fix(responses): run stream failure and success hooks on the iterating loop instead of blocking it by devin-ai-integration[bot] · Pull Request #43270 · BerriAI/litellm

devin-ai-integration · 2026-09-26T02:07:51Z

TLDR

Problem this solves:

A provider stream dropping mid-body on native /v1/responses stalls the whole proxy
The failure logging ran on a helper thread while the event loop sat waiting for it
It never returns when a callback needs the blocked loop, so the pod freezes until restart

How it solves it:

With a running loop, failure logging becomes a task on it, awaited before the error propagates
Sync failure callbacks are submitted after that async logging finishes, keeping the old order
The completed-stream success deployment hook is scheduled the same way
Without a running loop (the sync iterator) the blocking path is unchanged

User Flow

Before: a developer streaming from /v1/responses through a proxy with an async logging callback (Datadog here) freezes the proxy for everyone the moment one provider stream drops

They send POST https://litellm-domain/v1/responses with "stream": true and get response.created plus the first output deltas
The provider connection drops mid-body
Every other request, GET https://litellm-domain/health/liveliness included, times out for as long as the failure logging takes (20s with a slow Datadog agent), and the pod can stay frozen until it is restarted when the drop lands while another request's log is still being sent
Only after that does their stream get response.failed

After: the same drop reaches their client as response.failed while the proxy keeps serving

They send POST https://litellm-domain/v1/responses with "stream": true and get response.created plus the first output deltas
The provider connection drops mid-body
GET https://litellm-domain/health/liveliness keeps answering 200 throughout and the failure event still reaches Datadog
Their stream gets response.failed once the drop has been logged and ends

Linear ticket

Resolves LIT-8678

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Both legs ran the same rig on one laptop and differ only in the commit the proxy was booted from: the Before leg at the PR's merge base 4adbc13 and the After leg at the PR tip 0be4a64, each in its own worktree ($WT) with its own venv ($PY is $WT/.venv/bin/python, $MK the proxy master key). Every case boots one fresh proxy with --num_workers 2 on a random free port from rig/config.yaml, which carries three deployments: gpt-5.4-mini (openai/gpt-5.4-mini straight to api.openai.com), gpt-5.4-mini-flaky-upstream (the same model with api_base pointing at a local relay that forwards every request to https://api.openai.com and, once armed with touch rig/out/cut.signal, aborts the proxy's connection mid-body at the next chunk after 20 output deltas) and gpt-5.4-mini-bridge-flaky-upstream (openai/responses/gpt-5.4-mini through that relay, so /v1/chat/completions and /v1/messages ride the Responses API stream). litellm_settings.callbacks: ["datadog"] runs in agent mode (LITELLM_DD_AGENT_HOST/LITELLM_DD_AGENT_PORT) against a local stand-in that accepts every POST /api/v2/logs and holds its 202 for 20s, a slow Datadog agent, with DD_BATCH_SIZE=1 so every event flushes right away. Throughout each case six keep-alive HTTP/1.1 connections each send GET /health/liveliness once a second (what a Kubernetes liveness probe and every other caller of that pod see); a connection with no answer for 2s or more is listed as hung with how long it waited. The clients are the OpenAI Python SDK (client.responses.create(..., stream=True) on /v1/responses, client.chat.completions.create(..., stream=True) on /v1/chat/completions, a non-stream client.responses.create(...) for the requests whose log flush holds the logger) and the Anthropic Python SDK (client.messages.create(..., stream=True) on /v1/messages), all against the real gpt-5.4-mini. Response ids in the non-stream output are shortened to resp_<id>. Every timestamp is local wall clock, cases ran back to back

Before (`4adbc13`)

Mid-stream provider drop on /v1/responses

proxy booted 19:23:38: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 49660 --num_workers 2 (pid 22071, /health/liveliness answering)
19:24:07 cut armed (touch rig/out/cut.signal: the relay drops the upstream connection after 20 output deltas)
19:24:07 $PY rig/client_stream.py http://127.0.0.1:49660/v1 $MK gpt-5.4-mini-flaky-upstream

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

19:24:08 POST http://127.0.0.1:49660/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
19:24:10 response.created
19:24:10 response.in_progress
19:24:10 response.output_item.added
19:24:10 response.content_part.added
19:24:10 delta #1
19:24:10 delta #5
19:24:50 response.failed
19:24:50 stream ended normally after 10 deltas

Six keep-alive connections each sending GET http://127.0.0.1:49660/health/liveliness once a second throughout:

conn4: GET /health/liveliness sent 19:24:10 answered 200 after 19.6s
conn6: GET /health/liveliness sent 19:24:10 answered 200 after 19.6s
conn3: GET /health/liveliness sent 19:24:10 answered 200 after 19.6s
summary: 222 probes over 6 keep-alive connections; 3 connection(s) hung (conn3, conn4, conn6); slowest answered probe 19.6s

Upstream relay:

19:24:08 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
19:24:09 <- HTTP/1.1 200 OK
19:24:09 <- first response.output_text.delta passed through to the proxy
19:24:10 CUT: aborting the proxy's connection mid-body after 5665 bytes and 20 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:24:10 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:24:30 sent 202 for ['error']
19:24:30 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:24:50 sent 202 for ['error']

Result: cut_at=19:24:10 response.failed_at=19:24:50 cut_to_failed=40s response.completed_at=None probe=summary: 222 probes over 6 keep-alive connections; 3 connection(s) hung (conn3, conn4, conn6); slowest answered probe 19.6s

Drop while six other requests' log flushes hold the logger

proxy booted 19:25:04: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 25322 --num_workers 2 (pid 24987, /health/liveliness answering)
19:25:25 $PY rig/client_stream.py http://127.0.0.1:25322/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
19:25:27 stream is past its first delta; 6 concurrent non-stream requests: for i in 1 2 3 4 5 6; do $PY rig/client_once.py http://127.0.0.1:25322/v1 $MK gpt-5.4-mini & done; wait (each one's success log flush holds the Datadog logger's lock on its worker for 20s)
19:25:29 all 6 non-stream requests answered; cut armed (touch rig/out/cut.signal)

Non-stream clients:

19:25:27 POST http://127.0.0.1:25322/v1/responses stream=False model=gpt-5.4-mini
19:25:29 resp_<id> status=completed text='Hi there, friend.'
19:25:27 POST http://127.0.0.1:25322/v1/responses stream=False model=gpt-5.4-mini
19:25:28 resp_<id> status=completed text='Hi there, friend.'
19:25:27 POST http://127.0.0.1:25322/v1/responses stream=False model=gpt-5.4-mini
19:25:29 resp_<id> status=completed text='Hi there, friend.'
19:25:27 POST http://127.0.0.1:25322/v1/responses stream=False model=gpt-5.4-mini
19:25:28 resp_<id> status=completed text='Hi there, friend.'
19:25:27 POST http://127.0.0.1:25322/v1/responses stream=False model=gpt-5.4-mini
19:25:29 resp_<id> status=completed text='Hi there, friend.'
19:25:27 POST http://127.0.0.1:25322/v1/responses stream=False model=gpt-5.4-mini
19:25:28 resp_<id> status=completed text='Hi there!'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

19:25:26 POST http://127.0.0.1:25322/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
19:25:27 response.created
19:25:27 response.in_progress
19:25:27 response.output_item.added
19:25:27 response.content_part.added
19:25:27 delta #1
19:25:27 delta #5
19:25:27 delta #20
19:25:27 delta #50
19:26:09 response.failed
19:26:09 stream ended normally after 273 deltas

Six keep-alive connections each sending GET http://127.0.0.1:25322/health/liveliness once a second throughout:

summary: 282 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Upstream relay:

19:25:26 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
19:25:27 <- HTTP/1.1 200 OK
19:25:27 <- first response.output_text.delta passed through to the proxy
19:25:29 CUT: aborting the proxy's connection mid-body after 76086 bytes and 512 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:25:28 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:25:29 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:25:48 POST /api/v2/logs: 7 event(s) ['info', 'info', 'info', 'info', 'info', 'error', 'error']; holding the 202 for 20.0s
19:25:48 POST /api/v2/logs: 7 event(s) ['info', 'info', 'info', 'info', 'info', 'error', 'error']; holding the 202 for 20.0s
19:25:49 sent 202 for ['info']
19:25:49 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:25:49 POST /api/v2/logs: 7 event(s) ['info', 'info', 'info', 'info', 'info', 'error', 'error']; holding the 202 for 20.0s
19:25:49 POST /api/v2/logs: 7 event(s) ['info', 'info', 'info', 'info', 'info', 'error', 'error']; holding the 202 for 20.0s
19:26:08 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error', 'error']
19:26:08 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error', 'error']
19:26:09 sent 202 for ['info']
19:26:09 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error', 'error']
19:26:09 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error', 'error']

Result: cut_at=19:25:29 response.failed_at=19:26:09 cut_to_failed=40s response.completed_at=None probe=summary: 282 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Healthy stream completes

proxy booted 19:26:28: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 54144 --num_workers 2 (pid 28870, /health/liveliness answering)
19:26:58 no cut armed; $PY rig/client_stream.py http://127.0.0.1:54144/v1 $MK gpt-5.4-mini-flaky-upstream

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

19:26:58 POST http://127.0.0.1:54144/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
19:26:59 response.created
19:26:59 response.in_progress
19:26:59 response.output_item.added
19:26:59 response.content_part.added
19:26:59 delta #1
19:26:59 delta #5
19:27:00 delta #20
19:27:00 delta #50
19:27:05 response.output_text.done
19:27:05 response.content_part.done
19:27:05 response.output_item.done
19:27:05 response.completed
19:27:05 stream ended normally after 745 deltas

Six keep-alive connections each sending GET http://127.0.0.1:54144/health/liveliness once a second throughout:

summary: 66 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Upstream relay:

19:26:59 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
19:26:59 <- HTTP/1.1 200 OK
19:26:59 <- first response.output_text.delta passed through to the proxy
19:27:05 <- upstream stream ended normally after 221346 bytes and 1371 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:27:05 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s

Result: cut_at=None response.failed_at=None cut_to_failed=n/a response.completed_at=19:27:05 probe=summary: 66 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Mid-stream drop on /v1/chat/completions through the Responses bridge

proxy booted 19:27:22: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 56906 --num_workers 2 (pid 31663, /health/liveliness answering)
19:27:44 cut armed (touch rig/out/cut.signal: the relay drops the upstream connection after 20 output deltas)
19:27:44 $PY rig/client_stream_chat.py http://127.0.0.1:56906/v1 $MK gpt-5.4-mini-bridge-flaky-upstream

Streaming client (OpenAI Python SDK, client.chat.completions.create(..., stream=True) on a model served through the Responses API):

19:27:45 POST http://127.0.0.1:56906/v1/chat/completions stream=True model=gpt-5.4-mini-bridge-flaky-upstream
19:27:46 content chunk #1
19:27:46 content chunk #5
19:28:47 STREAM ERROR openai.APIError: litellm.APIConnectionError: APIConnectionError: OpenAIException - Response payload is not completed: <TransferEncodingError: 400, message='Not enough data to satisfy transfer length header.'>

Six keep-alive connections each sending GET http://127.0.0.1:56906/health/liveliness once a second throughout:

conn4: GET /health/liveliness sent 19:27:47 answered 200 after 19.5s
conn5: GET /health/liveliness sent 19:27:47 answered 200 after 19.5s
conn2: GET /health/liveliness sent 19:27:47 answered 200 after 19.5s
conn6: GET /health/liveliness sent 19:27:47 answered 200 after 19.5s
conn1: GET /health/liveliness sent 19:27:47 answered 200 after 19.5s
summary: 301 probes over 6 keep-alive connections; 5 connection(s) hung (conn1, conn2, conn4, conn5, conn6); slowest answered probe 19.5s

Upstream relay:

19:27:45 -> POST /v1/responses HTTP/1.1 (179 body bytes) forwarded to https://api.openai.com
19:27:46 <- HTTP/1.1 200 OK
19:27:46 <- first response.output_text.delta passed through to the proxy
19:27:47 CUT: aborting the proxy's connection mid-body after 5960 bytes and 20 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:27:47 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:28:07 sent 202 for ['error']
19:28:07 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:28:27 sent 202 for ['error']
19:28:27 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:28:47 sent 202 for ['error']

Result: cut_at=19:27:47 response.failed_at=19:28:47 cut_to_failed=60s response.completed_at=None probe=summary: 301 probes over 6 keep-alive connections; 5 connection(s) hung (conn1, conn2, conn4, conn5, conn6); slowest answered probe 19.5s

Mid-stream drop on /v1/messages through the Responses bridge

proxy booted 19:29:00: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 21091 --num_workers 2 (pid 36798, /health/liveliness answering)
19:29:25 cut armed (touch rig/out/cut.signal: the relay drops the upstream connection after 20 output deltas)
19:29:25 $PY rig/client_stream_messages.py http://127.0.0.1:21091 $MK gpt-5.4-mini-bridge-flaky-upstream

Streaming client (Anthropic Python SDK, client.messages.create(..., stream=True) on a model served through the Responses API):

19:29:26 POST http://127.0.0.1:21091/v1/messages stream=True model=gpt-5.4-mini-bridge-flaky-upstream
19:29:27 message_start
19:29:27 content_block_start
19:29:27 content_block_delta #1
19:29:27 content_block_delta #5
19:29:47 STREAM ERROR anthropic.APIStatusError: {'type': 'error', 'error': {'type': 'api_error', 'message': "Response payload is not completed: <TransferEncodingError: 400, message='Not enough data to satisfy transfer length header.'>"}}

Six keep-alive connections each sending GET http://127.0.0.1:21091/health/liveliness once a second throughout:

conn5: GET /health/liveliness sent 19:29:27 answered 200 after 19.9s
conn4: GET /health/liveliness sent 19:29:27 answered 200 after 19.9s
conn3: GET /health/liveliness sent 19:29:27 answered 200 after 19.9s
conn2: GET /health/liveliness sent 19:29:27 answered 200 after 19.9s
summary: 78 probes over 6 keep-alive connections; 4 connection(s) hung (conn2, conn3, conn4, conn5); slowest answered probe 19.9s

Upstream relay:

19:29:26 -> POST /v1/responses HTTP/1.1 (246 body bytes) forwarded to https://api.openai.com
19:29:27 <- HTTP/1.1 200 OK
19:29:27 <- first response.output_text.delta passed through to the proxy
19:29:27 CUT: aborting the proxy's connection mid-body after 5651 bytes and 20 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:29:27 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:29:47 sent 202 for ['error']

Result: cut_at=19:29:27 response.failed_at=19:29:47 cut_to_failed=20s response.completed_at=None probe=summary: 78 probes over 6 keep-alive connections; 4 connection(s) hung (conn2, conn3, conn4, conn5); slowest answered probe 19.9s

Drop while one earlier request's log flush holds the logger (the repro's frozen-pod sequence, three attempts)

Attempt 1

proxy booted 19:38:15: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 33495 --num_workers 2 (pid 61942, /health/liveliness answering)
19:38:56 $PY rig/client_stream.py http://127.0.0.1:33495/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
19:38:59 stream is past its first delta; one non-stream request: $PY rig/client_once.py http://127.0.0.1:33495/v1 $MK gpt-5.4-mini (its success log flush holds the Datadog logger's lock on its worker for 20s)
19:39:02 non-stream request answered; cut armed (touch rig/out/cut.signal); the stream client gets up to 150s from here

Non-stream client:

19:38:59 POST http://127.0.0.1:33495/v1/responses stream=False model=gpt-5.4-mini
19:39:01 resp_<id> status=completed text='Hi there, friend.'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

19:38:57 POST http://127.0.0.1:33495/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
19:38:59 response.created
19:38:59 response.in_progress
19:38:59 response.output_item.added
19:38:59 response.content_part.added
19:38:59 delta #1
19:38:59 delta #5
19:38:59 delta #20
19:38:59 delta #50
19:39:41 response.failed
19:39:41 stream ended normally after 401 deltas

Six keep-alive connections each sending GET http://127.0.0.1:33495/health/liveliness once a second throughout:

summary: 294 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.1s

Upstream relay:

19:38:58 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
19:38:58 <- HTTP/1.1 200 OK
19:38:58 <- first response.output_text.delta passed through to the proxy
19:39:02 CUT: aborting the proxy's connection mid-body after 110455 bytes and 747 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:39:01 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:39:21 POST /api/v2/logs: 3 event(s) ['info', 'error', 'error']; holding the 202 for 20.0s
19:39:21 sent 202 for ['info']
19:39:41 sent 202 for ['info', 'error', 'error']

Result: cut_at=19:39:02 response.failed_at=19:39:41 cut_to_failed=39s response.completed_at=None probe=summary: 294 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.1s

Attempt 2

proxy booted 19:39:58: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 21627 --num_workers 2 (pid 73627, /health/liveliness answering)
19:40:32 $PY rig/client_stream.py http://127.0.0.1:21627/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
19:40:35 stream is past its first delta; one non-stream request: $PY rig/client_once.py http://127.0.0.1:21627/v1 $MK gpt-5.4-mini (its success log flush holds the Datadog logger's lock on its worker for 20s)
19:40:37 non-stream request answered; cut armed (touch rig/out/cut.signal); the stream client gets up to 150s from here

Non-stream client:

19:40:35 POST http://127.0.0.1:21627/v1/responses stream=False model=gpt-5.4-mini
19:40:37 resp_<id> status=completed text='Hi there, friend.'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

19:40:34 POST http://127.0.0.1:21627/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
19:40:35 response.created
19:40:35 response.in_progress
19:40:35 response.output_item.added
19:40:35 response.content_part.added
19:40:35 delta #1
19:40:35 delta #5
19:40:35 delta #20
19:40:35 delta #50
19:41:17 response.failed
19:41:17 stream ended normally after 265 deltas

Six keep-alive connections each sending GET http://127.0.0.1:21627/health/liveliness once a second throughout:

summary: 294 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.1s

Upstream relay:

19:40:34 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
19:40:35 <- HTTP/1.1 200 OK
19:40:35 <- first response.output_text.delta passed through to the proxy
19:40:37 CUT: aborting the proxy's connection mid-body after 73929 bytes and 495 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:40:37 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:40:57 sent 202 for ['info']
19:40:57 POST /api/v2/logs: 3 event(s) ['info', 'error', 'error']; holding the 202 for 20.0s
19:41:17 sent 202 for ['info', 'error', 'error']

Result: cut_at=19:40:37 response.failed_at=19:41:17 cut_to_failed=40s response.completed_at=None probe=summary: 294 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.1s

Attempt 3

proxy booted 19:41:38: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 38155 --num_workers 2 (pid 80554, /health/liveliness answering)
19:42:43 $PY rig/client_stream.py http://127.0.0.1:38155/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
19:42:46 stream is past its first delta; one non-stream request: $PY rig/client_once.py http://127.0.0.1:38155/v1 $MK gpt-5.4-mini (its success log flush holds the Datadog logger's lock on its worker for 20s)
19:42:47 non-stream request answered; cut armed (touch rig/out/cut.signal); the stream client gets up to 150s from here

Non-stream client:

19:42:46 POST http://127.0.0.1:38155/v1/responses stream=False model=gpt-5.4-mini
19:42:47 resp_<id> status=completed text='Hi there, friend.'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

19:42:44 POST http://127.0.0.1:38155/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
19:42:45 response.created
19:42:45 response.in_progress
19:42:45 response.output_item.added
19:42:45 response.content_part.added
19:42:45 delta #1
19:42:45 delta #5
19:42:45 delta #20
19:42:46 delta #50
19:43:28 response.failed
19:43:28 stream ended normally after 345 deltas

Six keep-alive connections each sending GET http://127.0.0.1:38155/health/liveliness once a second throughout:

conn5: GET /health/liveliness sent 19:42:48 answered 200 after 19.7s
conn1: GET /health/liveliness sent 19:42:48 answered 200 after 19.7s
summary: 256 probes over 6 keep-alive connections; 2 connection(s) hung (conn1, conn5); slowest answered probe 19.7s

Upstream relay:

19:42:45 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
19:42:45 <- HTTP/1.1 200 OK
19:42:45 <- first response.output_text.delta passed through to the proxy
19:42:48 CUT: aborting the proxy's connection mid-body after 95300 bytes and 640 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

19:42:47 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:42:48 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:43:07 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
19:43:07 sent 202 for ['info']
19:43:08 sent 202 for ['error']
19:43:08 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
19:43:27 sent 202 for ['info']
19:43:28 sent 202 for ['error']

Result: cut_at=19:42:48 response.failed_at=19:43:28 cut_to_failed=40s response.completed_at=None probe=summary: 256 probes over 6 keep-alive connections; 2 connection(s) hung (conn1, conn5); slowest answered probe 19.7s

After (`0be4a64`)

Mid-stream provider drop on /v1/responses

proxy booted 21:37:05: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 23575 --num_workers 2 (pid 2497, /health/liveliness answering)
21:37:32 cut armed (touch rig/out/cut.signal: the relay drops the upstream connection after 20 output deltas)
21:37:32 $PY rig/client_stream.py http://127.0.0.1:23575/v1 $MK gpt-5.4-mini-flaky-upstream

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

21:37:33 POST http://127.0.0.1:23575/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
21:37:34 response.created
21:37:34 response.in_progress
21:37:34 response.output_item.added
21:37:34 response.content_part.added
21:37:34 delta #1
21:37:34 delta #5
21:38:15 response.failed
21:38:15 stream ended normally after 11 deltas

Six keep-alive connections each sending GET http://127.0.0.1:23575/health/liveliness once a second throughout:

summary: 276 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.0s

Upstream relay:

21:37:33 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
21:37:34 <- HTTP/1.1 200 OK
21:37:34 <- first response.output_text.delta passed through to the proxy
21:37:35 CUT: aborting the proxy's connection mid-body after 5942 bytes and 20 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:37:35 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:37:55 sent 202 for ['error']
21:37:55 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:38:15 sent 202 for ['error']

Result: cut_at=21:37:35 response.failed_at=21:38:15 cut_to_failed=40s response.completed_at=None probe=summary: 276 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.0s

Drop while six other requests' log flushes hold the logger

proxy booted 21:38:35: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 44750 --num_workers 2 (pid 10818, /health/liveliness answering)
21:39:08 $PY rig/client_stream.py http://127.0.0.1:44750/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
21:39:10 stream is past its first delta; 6 concurrent non-stream requests: for i in 1 2 3 4 5 6; do $PY rig/client_once.py http://127.0.0.1:44750/v1 $MK gpt-5.4-mini & done; wait (each one's success log flush holds the Datadog logger's lock on its worker for 20s)
21:39:12 all 6 non-stream requests answered; cut armed (touch rig/out/cut.signal)

Non-stream clients:

21:39:10 POST http://127.0.0.1:44750/v1/responses stream=False model=gpt-5.4-mini
21:39:11 resp_<id> status=completed text='Hi there, friend.'
21:39:10 POST http://127.0.0.1:44750/v1/responses stream=False model=gpt-5.4-mini
21:39:11 resp_<id> status=completed text='Hi there!'
21:39:10 POST http://127.0.0.1:44750/v1/responses stream=False model=gpt-5.4-mini
21:39:12 resp_<id> status=completed text='Hi there, friend.'
21:39:10 POST http://127.0.0.1:44750/v1/responses stream=False model=gpt-5.4-mini
21:39:12 resp_<id> status=completed text='Hi there, friend.'
21:39:10 POST http://127.0.0.1:44750/v1/responses stream=False model=gpt-5.4-mini
21:39:11 resp_<id> status=completed text='Hi there! Hello.'
21:39:10 POST http://127.0.0.1:44750/v1/responses stream=False model=gpt-5.4-mini
21:39:11 resp_<id> status=completed text='Hi there!'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

21:39:08 POST http://127.0.0.1:44750/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
21:39:10 response.created
21:39:10 response.in_progress
21:39:10 response.output_item.added
21:39:10 response.content_part.added
21:39:10 delta #1
21:39:10 delta #5
21:39:10 delta #20
21:39:10 delta #50
21:40:12 response.failed
21:40:12 stream ended normally after 388 deltas

Six keep-alive connections each sending GET http://127.0.0.1:44750/health/liveliness once a second throughout:

summary: 408 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.1s

Upstream relay:

21:39:09 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
21:39:09 <- HTTP/1.1 200 OK
21:39:10 <- first response.output_text.delta passed through to the proxy
21:39:13 CUT: aborting the proxy's connection mid-body after 107015 bytes and 704 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:39:11 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
21:39:12 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
21:39:31 POST /api/v2/logs: 6 event(s) ['info', 'info', 'info', 'info', 'info', 'error']; holding the 202 for 20.0s
21:39:31 POST /api/v2/logs: 6 event(s) ['info', 'info', 'info', 'info', 'info', 'error']; holding the 202 for 20.0s
21:39:31 POST /api/v2/logs: 6 event(s) ['info', 'info', 'info', 'info', 'info', 'error']; holding the 202 for 20.0s
21:39:32 POST /api/v2/logs: 6 event(s) ['info', 'info', 'info', 'info', 'info', 'error']; holding the 202 for 20.0s
21:39:32 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
21:39:32 sent 202 for ['info']
21:39:51 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error']
21:39:51 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error']
21:39:52 sent 202 for ['info', 'info', 'info', 'info', 'info', 'error']
21:39:52 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:39:52 sent 202 for ['info']
21:40:12 sent 202 for ['error']

Result: cut_at=21:39:13 response.failed_at=21:40:12 cut_to_failed=59s response.completed_at=None probe=summary: 408 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.1s

Healthy stream completes

proxy booted 21:40:37: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 32256 --num_workers 2 (pid 23446, /health/liveliness answering)
21:41:07 no cut armed; $PY rig/client_stream.py http://127.0.0.1:32256/v1 $MK gpt-5.4-mini-flaky-upstream

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

21:41:07 POST http://127.0.0.1:32256/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
21:41:09 response.created
21:41:09 response.in_progress
21:41:09 response.output_item.added
21:41:09 response.content_part.added
21:41:09 delta #1
21:41:09 delta #5
21:41:09 delta #20
21:41:09 delta #50
21:41:15 response.output_text.done
21:41:15 response.content_part.done
21:41:15 response.output_item.done
21:41:15 response.completed
21:41:15 stream ended normally after 767 deltas

Six keep-alive connections each sending GET http://127.0.0.1:32256/health/liveliness once a second throughout:

summary: 72 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.0s

Upstream relay:

21:41:08 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
21:41:08 <- HTTP/1.1 200 OK
21:41:08 <- first response.output_text.delta passed through to the proxy
21:41:15 <- upstream stream ended normally after 227502 bytes and 1422 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:41:15 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s

Result: cut_at=None response.failed_at=None cut_to_failed=n/a response.completed_at=21:41:15 probe=summary: 72 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.0s

Mid-stream drop on /v1/chat/completions through the Responses bridge

proxy booted 21:41:38: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 46622 --num_workers 2 (pid 28697, /health/liveliness answering)
21:42:08 cut armed (touch rig/out/cut.signal: the relay drops the upstream connection after 20 output deltas)
21:42:08 $PY rig/client_stream_chat.py http://127.0.0.1:46622/v1 $MK gpt-5.4-mini-bridge-flaky-upstream

Streaming client (OpenAI Python SDK, client.chat.completions.create(..., stream=True) on a model served through the Responses API):

21:42:08 POST http://127.0.0.1:46622/v1/chat/completions stream=True model=gpt-5.4-mini-bridge-flaky-upstream
21:42:09 content chunk #1
21:42:09 content chunk #5
21:43:09 STREAM ERROR openai.APIError: litellm.APIConnectionError: APIConnectionError: OpenAIException - Response payload is not completed: <TransferEncodingError: 400, message='Not enough data to satisfy transfer length header.'>

Six keep-alive connections each sending GET http://127.0.0.1:46622/health/liveliness once a second throughout:

summary: 390 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.0s

Upstream relay:

21:42:08 -> POST /v1/responses HTTP/1.1 (179 body bytes) forwarded to https://api.openai.com
21:42:09 <- HTTP/1.1 200 OK
21:42:09 <- first response.output_text.delta passed through to the proxy
21:42:09 CUT: aborting the proxy's connection mid-body after 6204 bytes and 23 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:42:09 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:42:29 sent 202 for ['error']
21:42:29 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:42:49 sent 202 for ['error']
21:42:49 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:43:09 sent 202 for ['error']

Result: cut_at=21:42:09 response.failed_at=21:43:09 cut_to_failed=60s response.completed_at=None probe=summary: 390 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.0s

Mid-stream drop on /v1/messages through the Responses bridge

proxy booted 21:43:17: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 57576 --num_workers 2 (pid 36647, /health/liveliness answering)
21:44:01 cut armed (touch rig/out/cut.signal: the relay drops the upstream connection after 20 output deltas)
21:44:01 $PY rig/client_stream_messages.py http://127.0.0.1:57576 $MK gpt-5.4-mini-bridge-flaky-upstream

Streaming client (Anthropic Python SDK, client.messages.create(..., stream=True) on a model served through the Responses API):

21:44:02 POST http://127.0.0.1:57576/v1/messages stream=True model=gpt-5.4-mini-bridge-flaky-upstream
21:44:04 message_start
21:44:04 content_block_start
21:44:04 content_block_delta #1
21:44:04 content_block_delta #5
21:44:24 STREAM ERROR anthropic.APIStatusError: {'type': 'error', 'error': {'type': 'api_error', 'message': "Response payload is not completed: <TransferEncodingError: 400, message='Not enough data to satisfy transfer length header.'>"}}

Six keep-alive connections each sending GET http://127.0.0.1:57576/health/liveliness once a second throughout:

summary: 156 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.1s

Upstream relay:

21:44:03 -> POST /v1/responses HTTP/1.1 (246 body bytes) forwarded to https://api.openai.com
21:44:03 <- HTTP/1.1 200 OK
21:44:03 <- first response.output_text.delta passed through to the proxy
21:44:04 CUT: aborting the proxy's connection mid-body after 5651 bytes and 20 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:44:04 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:44:24 sent 202 for ['error']

Result: cut_at=21:44:04 response.failed_at=21:44:24 cut_to_failed=20s response.completed_at=None probe=summary: 156 probes over 6 keep-alive connections; 0 connection(s) hung (none); 0 never answered; slowest answered probe 0.1s

Drop while one earlier request's log flush holds the logger (the repro's frozen-pod sequence, three attempts)

Attempt 1

proxy booted 21:51:59: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 23430 --num_workers 2 (pid 95090, /health/liveliness answering)
21:52:33 $PY rig/client_stream.py http://127.0.0.1:23430/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
21:52:34 stream is past its first delta; one non-stream request: $PY rig/client_once.py http://127.0.0.1:23430/v1 $MK gpt-5.4-mini (its success log flush holds the Datadog logger's lock on its worker for 20s)
21:52:36 non-stream request answered; cut armed (touch rig/out/cut.signal); the stream client gets up to 150s from here

Non-stream client:

21:52:35 POST http://127.0.0.1:23430/v1/responses stream=False model=gpt-5.4-mini
21:52:36 resp_<id> status=completed text='Hi there, friend.'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

21:52:33 POST http://127.0.0.1:23430/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
21:52:34 response.created
21:52:34 response.in_progress
21:52:34 response.output_item.added
21:52:34 response.content_part.added
21:52:34 delta #1
21:52:34 delta #5
21:52:34 delta #20
21:52:35 delta #50
21:53:36 response.failed
21:53:36 stream ended normally after 409 deltas

Six keep-alive connections each sending GET http://127.0.0.1:23430/health/liveliness once a second throughout:

summary: 402 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.1s

Upstream relay:

21:52:34 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
21:52:34 <- HTTP/1.1 200 OK
21:52:34 <- first response.output_text.delta passed through to the proxy
21:52:37 CUT: aborting the proxy's connection mid-body after 112447 bytes and 763 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:52:36 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
21:52:56 POST /api/v2/logs: 2 event(s) ['info', 'error']; holding the 202 for 20.0s
21:53:16 sent 202 for ['info', 'error']
21:53:16 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:53:36 sent 202 for ['error']

Result: cut_at=21:52:37 response.failed_at=21:53:36 cut_to_failed=59s response.completed_at=None probe=summary: 402 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.1s

Attempt 2

proxy booted 21:54:08: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 47366 --num_workers 2 (pid 9250, /health/liveliness answering)
21:54:48 $PY rig/client_stream.py http://127.0.0.1:47366/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
21:54:49 stream is past its first delta; one non-stream request: $PY rig/client_once.py http://127.0.0.1:47366/v1 $MK gpt-5.4-mini (its success log flush holds the Datadog logger's lock on its worker for 20s)
21:54:51 non-stream request answered; cut armed (touch rig/out/cut.signal); the stream client gets up to 150s from here

Non-stream client:

21:54:50 POST http://127.0.0.1:47366/v1/responses stream=False model=gpt-5.4-mini
21:54:51 resp_<id> status=completed text='Hi there, friend.'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

21:54:48 POST http://127.0.0.1:47366/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
21:54:49 response.created
21:54:49 response.in_progress
21:54:49 response.output_item.added
21:54:49 response.content_part.added
21:54:49 delta #1
21:54:49 delta #5
21:54:49 delta #20
21:54:50 delta #50
21:55:51 response.failed
21:55:51 stream ended normally after 390 deltas

Six keep-alive connections each sending GET http://127.0.0.1:47366/health/liveliness once a second throughout:

summary: 402 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Upstream relay:

21:54:49 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
21:54:49 <- HTTP/1.1 200 OK
21:54:49 <- first response.output_text.delta passed through to the proxy
21:54:52 CUT: aborting the proxy's connection mid-body after 107451 bytes and 725 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:54:51 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
21:55:11 POST /api/v2/logs: 2 event(s) ['info', 'error']; holding the 202 for 20.0s
21:55:31 sent 202 for ['info', 'error']
21:55:31 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:55:51 sent 202 for ['error']

Result: cut_at=21:54:52 response.failed_at=21:55:51 cut_to_failed=59s response.completed_at=None probe=summary: 402 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Attempt 3

proxy booted 21:56:00: cd $WT && .venv/bin/python litellm/proxy/proxy_cli.py --config rig/config.yaml --port 25140 --num_workers 2 (pid 19871, /health/liveliness answering)
21:56:35 $PY rig/client_stream.py http://127.0.0.1:25140/v1 $MK gpt-5.4-mini-flaky-upstream (no cut armed yet)
21:56:36 stream is past its first delta; one non-stream request: $PY rig/client_once.py http://127.0.0.1:25140/v1 $MK gpt-5.4-mini (its success log flush holds the Datadog logger's lock on its worker for 20s)
21:56:37 non-stream request answered; cut armed (touch rig/out/cut.signal); the stream client gets up to 150s from here

Non-stream client:

21:56:36 POST http://127.0.0.1:25140/v1/responses stream=False model=gpt-5.4-mini
21:56:37 resp_<id> status=completed text='Hi there, friend.'

Streaming client (OpenAI Python SDK, client.responses.create(..., stream=True)):

21:56:35 POST http://127.0.0.1:25140/v1/responses stream=True model=gpt-5.4-mini-flaky-upstream
21:56:36 response.created
21:56:36 response.in_progress
21:56:36 response.output_item.added
21:56:36 response.content_part.added
21:56:36 delta #1
21:56:36 delta #5
21:56:36 delta #20
21:56:36 delta #50
21:57:38 response.failed
21:57:38 stream ended normally after 298 deltas

Six keep-alive connections each sending GET http://127.0.0.1:25140/health/liveliness once a second throughout:

summary: 396 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Upstream relay:

21:56:35 -> POST /v1/responses HTTP/1.1 (103 body bytes) forwarded to https://api.openai.com
21:56:36 <- HTTP/1.1 200 OK
21:56:36 <- first response.output_text.delta passed through to the proxy
21:56:38 CUT: aborting the proxy's connection mid-body after 82724 bytes and 546 output_text.delta events

Datadog agent stand-in (holds every 202 for 20s):

21:56:37 POST /api/v2/logs: 1 event(s) ['info']; holding the 202 for 20.0s
21:56:57 sent 202 for ['info']
21:56:57 POST /api/v2/logs: 2 event(s) ['info', 'error']; holding the 202 for 20.0s
21:57:17 sent 202 for ['info', 'error']
21:57:17 POST /api/v2/logs: 1 event(s) ['error']; holding the 202 for 20.0s
21:57:37 sent 202 for ['error']

Result: cut_at=21:56:38 response.failed_at=21:57:38 cut_to_failed=60s response.completed_at=None probe=summary: 396 probes over 6 keep-alive connections; 0 connection(s) hung (none); slowest answered probe 0.0s

Observations from both legs, with whether this PR causes each, makes it worse, or leaves it alone:

response.failed lands ~40s after the cut on /v1/responses, both legs; PR leaves it alone
Cases 2 and 6 error 59-60s after the cut, was 40s; PR causes it
Chat bridge error 60s after the cut on both legs; PR leaves it alone
Messages bridge error 20s after the cut on both legs; PR leaves it alone
Before cases 1, 4, 5 hung 3, 5, 4 probe connections; PR removes that
Before cases 2, 6 logged loop-binding errors; PR removes them
Case 2 hung nothing on either leg
Client receives about half the provider's output delta count; PR leaves it alone
One dropped stream sends Datadog two or three error events; PR leaves it alone
Datadog received duplicate event batches under lock contention (case 2); PR leaves it alone
Dropped-stream error text is aiohttp's TransferEncodingError; PR leaves it alone
Case 5 logs one AnthropicResponsesStreamWrapper ERROR line, both legs; PR leaves it alone
Permanent freeze (repro leg 2, one worker) not seen two-worker

Type

🐛 Bug Fix

Decisions

A running loop is detected with asyncio.get_running_loop() rather than by iterator class, so a sync iterator driven from inside a live loop takes the non-blocking path as well, and with no loop the sync path is byte for byte what it was
The base's hook set is kept as it was (the async failure handler, then the sync failure_handler on the executor); only where and when they run changed. An earlier revision of this PR went through dispatch_failure_handlers(prefer_async_handlers=True), which only submits the sync handler when sync failure callbacks are configured; it was dropped because it changed which callbacks fire, not just when
On the async iterator the failure logging is awaited inside __anext__ before the exception propagates, instead of being fired and forgotten the way chat streaming's _log_stream_failure_and_raise does it. That keeps the base's guarantee that logging has completed by the time the caller sees the error, holds only this one request while it waits (every other request keeps being served), and means a client that goes away right after the error cannot cancel the logging out from under a failed-deployment attribution. A bounded wait was considered and rejected as a knob nobody asked for
The sync failure_handler is executor-submitted in a finally after the awaited async handler, so the base's order (async first, then sync) holds even when the async handler raises
A sync iterator iterated from inside a running loop (__next__ under a loop) cannot await, so it schedules the same task fire-and-forget; tasks live in a module-level strong-reference set with a done callback that logs a failed task at ERROR, where the base swallowed the exception
One end_time is captured once and handed to both handlers instead of each computing its own
The completed-stream success deployment hook gets the same treatment because it blocked the loop the same way from the same file and the stream discards its return value; on the async iterator it is awaited before StopAsyncIteration propagates, since that is an Exception and goes through the same except

Caveats (if any)

Low

The sync __next__ path with no running loop still blocks its own thread
- Unchanged from the base and nobody reported it, so widening the fix there is net negative
A sync iterator driven inside a running loop schedules its logging fire-and-forget
- A caller that closes that loop right away can cancel the task, and the done callback logs it at ERROR
- The async iterator awaits the task, so the proxy path is not exposed; awaiting inside __next__ needs a loop we do not own
response.failed waits for a stalled logging callback with no timeout
- Cases 2 and 6 land 59-60s after the cut, 40s on the base
- On purpose per Decision 3, and Greptile's P1 thread on it conceded; a bounded wait would drop failure attribution on router fallbacks
Mock and cached streaming iterators schedule the success hook but never await it
- Test-only surfaces, not driven; adding an await there changes test fixtures for no user-facing gain
Router cooldown after a mid-stream failure was not read back in QA
- The failure handler order is the base's, so cooldown behavior is inherited, not changed
Non-required CI reds at the tip match main or a fix newer than the merge base, one rerun spent (workflow f69f748c) and nothing ported here
- build_and_test: two test_implemented_namespace_case_modules_remain_importable ocr params, main job 2218790, fixed by test: fix stale and state-leaking tests red on scheduled CircleCI #43266
- using_litellm_on_windows: red on main job 2218756, green on main from 99655b6
- langfuse_logging_unit_tests: timeout with 0 failures, main jobs 2218773, 2220819, 2221956
- local_testing_part1: two bedrock test_get_model_info tests, main job 2218778, fixed by test: fix stale and state-leaking tests red on scheduled CircleCI #43266
- proxy_store_model_in_db_tests: test_e2e_langfuse_callbacks_in_db, main job 2218794
- unit: the 8 PR-only reds are fixed by fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts #43294 and pass locally on the merged tree, the rest match main
- integration-extensions: test_passthrough_worker_sigkill_leaves_sibling_serving_and_logging, main job 2218749, rewritten by fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts #43294
- misc / Run tests: test_price_map_has_no_duplicate_keys, fixed by fix(cost-map): remove duplicate openrouter/perceptron/perceptron-mk1.5 entry #43273

Live A/B risk check

Verdict: no dependent path breaks. Three legs ran the same 33 scenarios (28 proxy runs plus 5 SDK runs) against the real gpt-5.4-mini through a forwarding relay, each as one proxy with --num_workers 2 in its own worktree and venv against one Postgres database: Before at the merge base 4adbc13 (proxy port 35950, relay 21618), After at the PR tip 0be4a64 (33108, 52916), and Merged at dcc6517d71, the tip merged into main 5856a36307 (47492, 25310). Every leg printed its worktree, its loaded litellm.__file__, and whether the head-only symbols _spawn_logging_task and _await_pending_logging import (absent on Before, present on After and Merged). The rig loaded one consumer of every hook kind through the documented extension points: a CustomLogger with only sync hooks, one with only async hooks, a CustomGuardrail whose async_post_call_success_deployment_hook raises on purpose, a legacy failure_callback callable, and a legacy success_callback callable, each appending a row (hook, thread, loop, request keys) per call; spend rows were read back through GET /spend/logs by request tag

Breaking: none observed. Per scenario the hook call counts are identical on all three legs (87 callback rows per leg; a mid-stream cut on /v1/responses records async_log_failure_event once and the legacy failure callable once on each leg, an early cut with router fallback records the failure hooks plus the three success hooks on each leg, and the SDK sync iterator under a running loop records the async, legacy, and sync failure hooks once each on each leg, including the variant that exits the interpreter right after the failure). The relay forwarded and cut the same requests on every leg (6 GET /v1/models, 29 POST /v1/responses, the same cut mode per marker, request bodies the same size Before vs Merged and 1 byte smaller on After from the marker text). Spend rows are 1 per marker per run on every leg. Client verdicts differ only in call ids, marker text, and output length

Backward incompatible:

Under a held logger lock the client sees response.failed later: QA case 2 at 59s after the cut and case 6 at 59s, 59s, 60s, against 40s and 39s, 40s, 40s on the base, because the failure logging is now awaited before the error propagates (Decision 3, commit f1407ac); shipped on purpose so the error never outruns its log
Async failure and success callbacks now run on the serving loop's thread instead of a helper thread with a private loop: the callback rows show ThreadPoolExecutor-N_0 on Before and MainThread on After and Merged for every async hook. This is the fix itself (commit a2ac961); a callback bound to the serving loop now works, and one that blocks its thread blocks a loop it already blocked before
A raising success deployment hook or logging task is now logged: LiteLLM:ERROR: streaming_iterator.py:169 - Responses stream post-call success hook failed: ... appears once on After and Merged for the raising-guardrail scenario and never on Before, which swallowed it (commit a2ac961, Decision 5)

Regression risk (dependents this run did not reach):

The sync iterator with no running loop at all (a plain script calling litellm.responses(stream=True)): its code path is the base's, unchanged by the diff, but no leg drove it; it takes a scripted SDK run outside asyncio with the relay cut armed
Consumer cancellation during the await: a client that disconnects while its failure logging is being awaited leaves the task running (the strong-reference set keeps it), but no leg cut the client during the 20s flush; it takes closing the client socket mid-hold and reading the stand-in's log for the event
The mock and cached streaming iterators inherit the success hook scheduling and never await it, so their hook is now fire-and-forget where it blocked before; not driven
Router cooldown from a mid-stream failure was not read back after the cut; the early-cut scenario did complete through the fallback deployment on every leg, with the same spend row model on each

Dependency graph (dependents of _handle_failure, _run_post_success_hooks, ResponsesAPIStreamingIterator.__anext__, __next__, and the new task helpers):

/v1/responses streaming through the proxy (mid-stream cut, early cut with fallback, clean EOF without a terminal event, healthy stream, healthy stream with a raising guardrail): verified-live, and tested by the five unit tests in tests/unit/responses/test_streaming_iterator.py
/v1/chat/completions through the Responses bridge (openai/responses/...), mid-stream cut: verified-live (QA case 4 as well)
/v1/messages through the Responses bridge, mid-stream cut: verified-live (QA case 5 as well)
Non-streaming /v1/responses: verified-live, unchanged
SDK litellm.aresponses(stream=True) under asyncio.run, exiting right after the failure and with a 3s settle: verified-live
SDK sync litellm.responses(stream=True) iterated inside a running loop, exiting right after, with a 3s settle and a legacy failure callback, and with only CustomLogger instances: verified-live
SDK sync iterator with no running loop: untested
Callback kinds (async CustomLogger, sync-only CustomLogger, legacy failure and success callables, guardrail post-call success deployment hook, the Datadog integration under flush_lock contention in QA cases 2 and 6): verified-live
Spend rows and request tags: verified-live
GET /health/liveliness on six keep-alive connections during a failure: verified-live (0 hung on After in every case; 3, 5, and 4 hung on Before in cases 1, 4, and 5)
Mock and cached streaming iterators: untested
Router deployment cooldown after a mid-stream failure: untested
Dashboard and API clients: unreachable, no response field or wire contract changed

Not verified:

Outbound request bodies beyond their size: the relay recorded the request line, byte count, and cut mode, not the body key set or values
Hostile-input cells: the diff reads no new header or body field, so none apply
The real Datadog intake: every leg used a local stand-in that holds each 202 for 20s, which is what reproduces the reported freeze; nothing was read back from Datadog itself
The bridge paths' callback rows: the recorder keys rows by a marker inside input or messages, and the bridge kwargs carry neither on any leg, so the chat and messages bridge scenarios were compared on client verdict, spend rows, and relay counts only
The sync no-loop iterator, the mock and cached iterators, router cooldown, and a mid-await client disconnect, as listed under regression risk

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
0be4a64 passes /live-pr-risk

Note

Medium Risk
Changes async hook scheduling and error propagation timing on the proxy’s hot /v1/responses streaming path; incorrect ordering or awaiting could regress logging, fallbacks, or health under load.

Overview
Fixes LIT-8678, where mid-stream /v1/responses failures could block the proxy event loop because failure and post-success deployment hooks ran via run_async_function on a helper thread while the iterating loop waited.

When asyncio.get_running_loop() is active, those hooks are create_task'd on that loop, tracked on the iterator, and the async ResponsesAPIStreamingIterator awaits them in __anext__ before the error propagates. Async failure logging still runs before sync failure_handler (executor), and a single end_time is shared. The completed-stream success deployment hook uses the same scheduling; with no running loop, behavior stays on the blocking path.

Unit tests assert hooks run on the test loop, failure attribution finishes before the consumer sees the error (router fallback), and sync-only / ordering guarantees when the sync iterator runs under a loop.

^{Reviewed by Cursor Bugbot for commit 0be4a64. Bugbot is set up for automated code reviews on this repo. Configure here.}

… loop instead of blocking it A dropped provider stream on native /v1/responses ran the failure logging through run_async_function from inside the async iterator, which parks the event loop thread on a helper-thread future until every failure callback returns, and never returns when a callback waits on state only that loop can advance. With a running loop the failure handlers (and the completed-stream success deployment hook) are now scheduled as tasks on it, the way chat streaming already does; the sync iterator keeps its blocking path

devin-ai-integration · 2026-09-26T02:07:54Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

greptile-apps · 2026-09-26T02:10:54Z

Confidence Score: 5/5

[Medium risk] Changes how stream completion hooks run in the iterator.

The PR appears safe to merge based on the current changes and review history

Findings

Logging delays stream termination ▶

Summary

The PR moves Responses stream failure logging and the completed-stream deployment hook onto the iterating event loop, then waits for failure logging before propagating async stream errors. The latest changes sequence sync failure callbacks after async logging and add an ordering test

_{Reviews (5) · Last reviewed commit: "fix(responses): submit the sync failure ..."}

codecov · 2026-09-26T02:14:04Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

codspeed · 2026-09-26T02:20:25Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_responses_stream_failure_handlers_on_loop (0be4a64) with main (d08746f)¹}

No successful run was found on main (4eb340a) during the generation of this report, so d08746f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

mateo-berri · 2026-09-26T02:29:43Z

@greptileai

…ting loop before propagating Keep the merge-base hook set for the native Responses stream: async_failure_handler plus the executor-thread failure_handler on failure, and the post-call success deployment hook on completion. Inside a running loop the async handler is scheduled as a task on that loop and the async iterator awaits it before re-raising, so the loop is never blocked on a foreign-loop future and the failure is attributed before the router's fallback wrapper re-enters the same logging object. The sync iterator inside a running loop keeps the task fire-and-forget with a strong reference.

mateo-berri · 2026-09-26T04:13:06Z

@greptileai

greptile-apps · 2026-09-26T04:15:59Z

+        except Exception:
+            await self._await_pending_logging()
+            raise


Logging delays stream termination If a logging callback is slow or stalls, this wait has no timeout, so the client cannot receive a provider error and the router cannot start a mid-stream fallback until logging finishes. It also catches normal StopAsyncIteration, making a successful stream wait for its success hook before ending. The event loop remains responsive, but the affected client can still be left waiting.

Base blocked the whole loop for the same duration, so this is parity. Awaiting also keeps failed-deployment attribution correct during router fallbacks

You're right. The important distinction is that this await yields to the iterating event loop; it no longer blocks the loop as the old helper-thread path did.

Waiting before propagating the exception is intentional: it preserves the existing ordering and ensures the primary deployment's failure is logged before router fallback can reuse or mutate the logging object. The added regression test also codifies that attribution requirement.

This can still delay the affected consumer if a callback never completes, but that is a per-stream delay rather than a proxy-wide event-loop stall. I’m withdrawing the original finding.

…one finishes

mateo-berri · 2026-09-26T04:34:10Z

@greptileai

mateo-berri · 2026-09-26T19:09:00Z

@greptileai

mateo-berri · 2026-09-26T19:16:11Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0be4a64. Configure here.}

mateo-berri

LGTM

…105.0) (#866) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.104.0` → `v1.105.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.105.0`](https://github.com/BerriAI/litellm/releases/tag/v1.105.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.104.3...v1.105.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.105.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.105.0/cosign.pub \ ghcr.io/berriai/litellm:v1.105.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - feat(mcp): share compatibility-aware result conversion across tool surfaces by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43089](https://github.com/BerriAI/litellm/pull/43089) - fix(caching): propagate auth cache invalidation over Redis Cluster via a node-level pub/sub client by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43110](https://github.com/BerriAI/litellm/pull/43110) - feat(model-catalog): add Rust registry validation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43136](https://github.com/BerriAI/litellm/pull/43136) - test(rust): reorganize core crate tests and split cache and OCR suites by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43177](https://github.com/BerriAI/litellm/pull/43177) - fix(proxy): give SpendLogToolIndex its own share of the cleanup budget and log a per-run summary by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41768](https://github.com/BerriAI/litellm/pull/41768) - feat(rust): hand upstream response headers to the native Messages stream by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43178](https://github.com/BerriAI/litellm/pull/43178) - feat(testkit): agent clients for Claude Code, Codex and opencode by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43181](https://github.com/BerriAI/litellm/pull/43181) - ci: run migrated unit selections on every event in legacy GHA shards by [@yuneng-berri](https://github.com/yuneng-berri) in [#43182](https://github.com/BerriAI/litellm/pull/43182) - test: move tests/test_litellm root and small trees into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43186](https://github.com/BerriAI/litellm/pull/43186) - fix(sso): gate /sso/debug routes behind ENABLE_SSO_DEBUG, off by default by [@ojensen-berri](https://github.com/ojensen-berri) in [#43150](https://github.com/BerriAI/litellm/pull/43150) - refactor(framer): replace Framer trait with tokio-util codecs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43193](https://github.com/BerriAI/litellm/pull/43193) - fix(anthropic): surface Responses bridge stream failures as Anthropic error events by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43126](https://github.com/BerriAI/litellm/pull/43126) - test: move tests/test_litellm/llms into tests/unit/llms by [@yuneng-berri](https://github.com/yuneng-berri) in [#43191](https://github.com/BerriAI/litellm/pull/43191) - test: move tests/test_litellm integrations and secret_managers into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43194](https://github.com/BerriAI/litellm/pull/43194) - feat(mcp): configure protocol versions and capability discovery by [@joshua-berri](https://github.com/joshua-berri) in [#43169](https://github.com/BerriAI/litellm/pull/43169) - fix(proxy): record streamed /v1/responses container ownership before the response.completed frame by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43140](https://github.com/BerriAI/litellm/pull/43140) - fix(tests): drop the repeated UNIT_FLAG key in test_unit_shard_missing_paths by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43212](https://github.com/BerriAI/litellm/pull/43212) - fix(router): count provider budget spend on every API surface by [@daqiangganjun](https://github.com/daqiangganjun) in [#38172](https://github.com/BerriAI/litellm/pull/38172) - fix(responses): fall back on pre-output stream drops, fail truncated streams, honor request_timeout by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43133](https://github.com/BerriAI/litellm/pull/43133) - fix(sentry): scrub PII and secrets inside object reprs and nested locals, add SENTRY_SEND_DEFAULT_PII opt-in by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43123](https://github.com/BerriAI/litellm/pull/43123) - fix(bedrock): surface a converse-stream 200 that decodes to no events as a 502 instead of an empty turn by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43213](https://github.com/BerriAI/litellm/pull/43213) - fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43206](https://github.com/BerriAI/litellm/pull/43206) - feat(integrations): add Databricks Zerobus trace logging callback by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42013](https://github.com/BerriAI/litellm/pull/42013) - fix(vertex_ai): keep legacy bucket_name in credential resolution and add GCS_BATCH_BUCKET_NAME env var by [@mubashir1osmani](https://github.com/mubashir1osmani) in [#42803](https://github.com/BerriAI/litellm/pull/42803) - feat(xai): add native xAI batches and files support by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42812](https://github.com/BerriAI/litellm/pull/42812) - fix(proxy): send a real error event when a /v1/messages stream fails by [@4refael](https://github.com/4refael) in [#41826](https://github.com/BerriAI/litellm/pull/41826) - fix(logging): redact raw_request when turn_off_message_logging is set in the proxy config by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43219](https://github.com/BerriAI/litellm/pull/43219) - fix(router): parse the classifier verdict out of surrounding prose instead of falling to the default tier by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43215](https://github.com/BerriAI/litellm/pull/43215) - test(zerobus): move tests into active CI selection by [@tin-berri](https://github.com/tin-berri) in [#43235](https://github.com/BerriAI/litellm/pull/43235) - fix(proxy): keep the submitted body out of 422 validation errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43231](https://github.com/BerriAI/litellm/pull/43231) - test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43199](https://github.com/BerriAI/litellm/pull/43199) - chore(cost-map): add fireworks us-only deepseek v4.1 flash priority prices by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43247](https://github.com/BerriAI/litellm/pull/43247) - chore(cost-map): sync openrouter prices and add perceptron-mk1.5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43246](https://github.com/BerriAI/litellm/pull/43246) - fix(cost-map): correct fireworks_ai deepseek-v4p1-flash pricing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43253](https://github.com/BerriAI/litellm/pull/43253) - feat(openrouter): add typesafe/jev-router to the cost map by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43248](https://github.com/BerriAI/litellm/pull/43248) - chore(cost-map): add fireworks priority prices for muse glimmer 30b and deepseek v4 flash vision exp by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43252](https://github.com/BerriAI/litellm/pull/43252) - refactor(rust): move credential inheritance and SDK limits into a driver preflight by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43259](https://github.com/BerriAI/litellm/pull/43259) - fix(router): hold Responses lifecycle events until output so a pre-output fallback announces one response by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43238](https://github.com/BerriAI/litellm/pull/43238) - refactor(http): hand out an owned Client and route all providers through the pool by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43245](https://github.com/BerriAI/litellm/pull/43245) - test: fix stale and state-leaking tests red on scheduled CircleCI by [@yuneng-berri](https://github.com/yuneng-berri) in [#43266](https://github.com/BerriAI/litellm/pull/43266) - fix(cost-map): retirement dates, chatgpt reasoning flags, bing pricing, bedrock mantle and mythos, azure gpt-5.6 alias, anthropic batch rates, new nebius, openrouter and xai rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42951](https://github.com/BerriAI/litellm/pull/42951) - refactor(rust): promote anthropic messages out of experimental_pass_through by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43269](https://github.com/BerriAI/litellm/pull/43269) - test: stop CI tests from downloading tokenizer files and images by [@yuneng-berri](https://github.com/yuneng-berri) in [#43257](https://github.com/BerriAI/litellm/pull/43257) - fix(rust): preserve nested optional import failures by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43265](https://github.com/BerriAI/litellm/pull/43265) - ci: drop main and litellm\_\* branch filters from the CircleCI litellm-main workflows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43272](https://github.com/BerriAI/litellm/pull/43272) - feat(proxy): email alerts at configured percentages of a team member budget by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42665](https://github.com/BerriAI/litellm/pull/42665) - fix(callbacks-legacy-python): traverse and release the retained headers dict by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43274](https://github.com/BerriAI/litellm/pull/43274) - test(integration): port langfuse callbacks-in-db coverage to the local harness by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43282](https://github.com/BerriAI/litellm/pull/43282) - test(e2e): accept the otel cost write as a linked root trace by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42931](https://github.com/BerriAI/litellm/pull/42931) - test: finish the non-proxy half of tests/test_litellm by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43281](https://github.com/BerriAI/litellm/pull/43281) - refactor(rust): prepare inference and auth foundations for the gateway by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43287](https://github.com/BerriAI/litellm/pull/43287) - feat(rust): add config, router and gateway crates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43289](https://github.com/BerriAI/litellm/pull/43289) - fix(cost-map): remove duplicate openrouter/perceptron/perceptron-mk1.5 entry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43273](https://github.com/BerriAI/litellm/pull/43273) - fix(e2e): resolve the blank-S3 gateway repo root from the litellm package location by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42911](https://github.com/BerriAI/litellm/pull/42911) - test(integration): run the Langfuse DB-callback test on its own scratch database by [@yuneng-berri](https://github.com/yuneng-berri) in [#43288](https://github.com/BerriAI/litellm/pull/43288) - refactor(rust): expand logging and test coverage across gateway and Anthropic messages by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43295](https://github.com/BerriAI/litellm/pull/43295) - refactor: daily fresh tech debt cleanup, rolling PR (2026-09-25) by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43151](https://github.com/BerriAI/litellm/pull/43151) - test(proxy_behavior): scope the management proxy fixture to its package so its spend monitor cannot race the spend tests by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43302](https://github.com/BerriAI/litellm/pull/43302) - fix(cost-map): registry audit 2026-09-26, MAI-Image-2.5-Flash price, Databricks Claude Opus 5.5, Azure Foundry retirement dates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43254](https://github.com/BerriAI/litellm/pull/43254) - fix(cost-map): price fireworks deepseek v4.1 flash at the prices api value by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43311](https://github.com/BerriAI/litellm/pull/43311) - fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts by [@yuneng-berri](https://github.com/yuneng-berri) in [#43294](https://github.com/BerriAI/litellm/pull/43294) - fix(otel): detach post-response service spans by request phase, name redis spans by operation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43237](https://github.com/BerriAI/litellm/pull/43237) - test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test by [@yuneng-berri](https://github.com/yuneng-berri) in [#43321](https://github.com/BerriAI/litellm/pull/43321) - test(integration): pin the team-admin status-code matrix across every management route by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43249](https://github.com/BerriAI/litellm/pull/43249) - feat(proxy): add fail_closed_rate_limit_enforcement to reject requests with 503 while Redis rate limit counters are unreachable by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43251](https://github.com/BerriAI/litellm/pull/43251) - fix(responses): run stream failure and success hooks on the iterating loop instead of blocking it by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43270](https://github.com/BerriAI/litellm/pull/43270) - fix(guardrails): block private destinations in custom code http_request and bound guardrail execution time by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43280](https://github.com/BerriAI/litellm/pull/43280) - feat(sail): add Sail as a provider with service_tier mapped to its completion window by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42840](https://github.com/BerriAI/litellm/pull/42840) - test(mcp): pin server resolution and authorization behavior by [@joshua-berri](https://github.com/joshua-berri) in [#43261](https://github.com/BerriAI/litellm/pull/43261) - refactor(anthropic): rename experimental_pass_through to pass_through by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43329](https://github.com/BerriAI/litellm/pull/43329) - refactor(mcp): add shared server resolver without changing callers by [@joshua-berri](https://github.com/joshua-berri) in [#43262](https://github.com/BerriAI/litellm/pull/43262) - ci: fail on new unbounded SQL IN lists and add a Prisma chunking helper by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42629](https://github.com/BerriAI/litellm/pull/42629) - test(logging): drain the logging worker after each logging callback test so no later test inherits its events by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43344](https://github.com/BerriAI/litellm/pull/43344) - feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43271](https://github.com/BerriAI/litellm/pull/43271) - fix(mcp): adopt shared server resolution and caller authorization by [@joshua-berri](https://github.com/joshua-berri) in [#43263](https://github.com/BerriAI/litellm/pull/43263) - fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43022](https://github.com/BerriAI/litellm/pull/43022) - feat(proxy): add maximum_daily_tag_spend_retention_period cleanup setting by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#39221](https://github.com/BerriAI/litellm/pull/39221) - fix(cost-map): sync OpenRouter, Together, Cohere and Azure AI registry values with official sources by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43337](https://github.com/BerriAI/litellm/pull/43337) - docs(pr-template): add the backport-stable label only for a P0 regression by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43351](https://github.com/BerriAI/litellm/pull/43351) - fix(params): keep *litellm*\* kwargs out of provider request bodies by construction by [@shrey-berri](https://github.com/shrey-berri) in [#43221](https://github.com/BerriAI/litellm/pull/43221) - test: remove substring guard test_default_api_base by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43355](https://github.com/BerriAI/litellm/pull/43355) - ci: cut CircleCI wall time without loosening test isolation by [@yuneng-berri](https://github.com/yuneng-berri) in [#43347](https://github.com/BerriAI/litellm/pull/43347) - fix(cost-map): correct azure gpt-4o-mini tts, transcribe, alias and MAI-Image-2.5 prices by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43357](https://github.com/BerriAI/litellm/pull/43357) - fix(tests): stop VCR recording and replaying a test's own localhost upstream by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43346](https://github.com/BerriAI/litellm/pull/43346) - test(integration): group /v1/messages contracts under tests/integration/messages_endpoint by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43352](https://github.com/BerriAI/litellm/pull/43352) - fix(params): validate stream_chunk_size once, before any provider call by [@shrey-berri](https://github.com/shrey-berri) in [#43222](https://github.com/BerriAI/litellm/pull/43222) - fix(openai): exclude fine-tuned and custom gpt-5-chat aliases from gpt-5 reasoning path by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43185](https://github.com/BerriAI/litellm/pull/43185) - fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43322](https://github.com/BerriAI/litellm/pull/43322) - fix(caching): stand default cache points down when extra_body hides a direct client mark by [@yuneng-berri](https://github.com/yuneng-berri) in [#43341](https://github.com/BerriAI/litellm/pull/43341) - fix(mcp): report reachability without stored credentials by [@tin-berri](https://github.com/tin-berri) in [#43240](https://github.com/BerriAI/litellm/pull/43240) - fix(mcp): align hub publication status and controls by [@tin-berri](https://github.com/tin-berri) in [#43241](https://github.com/BerriAI/litellm/pull/43241) - chore(cost-map): sync openrouter prices for deepseek, minimax, qwen and glm rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43384](https://github.com/BerriAI/litellm/pull/43384) - chore(cost-map): drop stale cache hit field from openrouter deepseek-v4-pro-0813 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43389](https://github.com/BerriAI/litellm/pull/43389) - fix(responses): stream guardrail pre-call block as SSE with a typed output item by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42507](https://github.com/BerriAI/litellm/pull/42507) - feat(e2e): read management routes back from the control plane replicas by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43373](https://github.com/BerriAI/litellm/pull/43373) - feat(rust): add the openai_like chat config foundation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43379](https://github.com/BerriAI/litellm/pull/43379) - feat(router): opt in to prompt-cache cost routing by [@tin-berri](https://github.com/tin-berri) in [#43232](https://github.com/BerriAI/litellm/pull/43232) - feat(otel): add SigNoz preset for OpenTelemetry v2 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43296](https://github.com/BerriAI/litellm/pull/43296) - feat(cli): reuse saved agent setup and add reconfigure by [@tin-berri](https://github.com/tin-berri) in [#43392](https://github.com/BerriAI/litellm/pull/43392) - fix(streaming): keep litellm Usage on text-completion usage chunks by [@yuneng-berri](https://github.com/yuneng-berri) in [#43047](https://github.com/BerriAI/litellm/pull/43047) - fix(vertex_ai): consider tools when validating context caching min tokens by [@agustin18](https://github.com/agustin18) in [#43319](https://github.com/BerriAI/litellm/pull/43319) - fix(anthropic): drop thinking blocks with empty thinking text, not just missing signature by [@shoemoney](https://github.com/shoemoney) in [#38049](https://github.com/BerriAI/litellm/pull/38049) - test(e2e): report batch cleanup leftovers as a plain UserWarning by [@yuneng-berri](https://github.com/yuneng-berri) in [#43405](https://github.com/BerriAI/litellm/pull/43405) - fix(tools): salvage concatenated JSON tool call arguments by [@kumarpriyanshu09](https://github.com/kumarpriyanshu09) in [#43260](https://github.com/BerriAI/litellm/pull/43260) - fix(gemini): forward seed to the Gemini API instead of rejecting it by [@YaseenBashaT](https://github.com/YaseenBashaT) in [#43197](https://github.com/BerriAI/litellm/pull/43197) - fix(vertex_ai): make Gemma fake streams work with traced Responses by [@stewartpark](https://github.com/stewartpark) in [#43147](https://github.com/BerriAI/litellm/pull/43147) - fix(responses): emit the reasoning item on streaming /v1/responses for signature-only thinking by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43414](https://github.com/BerriAI/litellm/pull/43414) - fix(anthropic): forward the per-turn-control beta to Azure AI Foundry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43415](https://github.com/BerriAI/litellm/pull/43415) - test(vertex_ai): assert the usage the Gemma responses stream actually reports by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43422](https://github.com/BerriAI/litellm/pull/43422) - fix(bedrock): keep the provider status code on unprocessable image errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43416](https://github.com/BerriAI/litellm/pull/43416) - fix(vertex_ai): stop importing the vertexai SDK in partner-model completion by [@anmolg1997](https://github.com/anmolg1997) in [#42274](https://github.com/BerriAI/litellm/pull/42274) - fix(token_counter): count Gemini function_declarations tools by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43417](https://github.com/BerriAI/litellm/pull/43417) - refactor(types): replace Any with proven types in 5 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43304](https://github.com/BerriAI/litellm/pull/43304) - chore(cost-map): update azure_ai/grok-4.6 input price from Azure pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43440](https://github.com/BerriAI/litellm/pull/43440) - chore(cost-map): add azure_ai/MAI-Cyber-1-Flash by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43446](https://github.com/BerriAI/litellm/pull/43446) - refactor(rust): share anthropic types, request helpers, and streaming contracts across crates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43426](https://github.com/BerriAI/litellm/pull/43426) - feat(rust): expand gateway configuration parsing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43460](https://github.com/BerriAI/litellm/pull/43460) - refactor(rust): share call lifecycle across route-owned inference by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43461](https://github.com/BerriAI/litellm/pull/43461) - feat(rust): add the HTTP host driver by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43462](https://github.com/BerriAI/litellm/pull/43462) - build(rust): package the gateway container by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43471](https://github.com/BerriAI/litellm/pull/43471) - refactor(rust): use shared execution in gateway inference by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43463](https://github.com/BerriAI/litellm/pull/43463) - fix(cost-map): add deprecation_date to together_ai Salesforce/Llama-Rank-V1 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43507](https://github.com/BerriAI/litellm/pull/43507) - feat(rust): support the HTTP Responses API by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43464](https://github.com/BerriAI/litellm/pull/43464) - fix(cost-map): sync openrouter prices from the models API by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43506](https://github.com/BerriAI/litellm/pull/43506) - fix(cost-map): set together_ai gpt-oss-20b and gemma-4-31B-it deprecation_date to 2026-09-15 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43509](https://github.com/BerriAI/litellm/pull/43509) - feat(rust): add litellm-db and litellm-db-testing workspace scaffolding by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43504](https://github.com/BerriAI/litellm/pull/43504) - feat(rust): connect Python inference bindings to shared routes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43465](https://github.com/BerriAI/litellm/pull/43465) - feat(rust): add structured route lifecycle tracing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43466](https://github.com/BerriAI/litellm/pull/43466) - feat(rust): separate gateway authentication and authorization by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43467](https://github.com/BerriAI/litellm/pull/43467) - feat(rust): add virtual key storage contracts by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43468](https://github.com/BerriAI/litellm/pull/43468) - feat(rust): add gateway UI login and sessions by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43469](https://github.com/BerriAI/litellm/pull/43469) - feat(rust): add the MCP gateway by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43470](https://github.com/BerriAI/litellm/pull/43470) - refactor(rust): remove delivery routing abstraction by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43514](https://github.com/BerriAI/litellm/pull/43514) - feat(azure): add mistral ocr pricing and azure max output limits by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43530](https://github.com/BerriAI/litellm/pull/43530) - refactor: clean up fresh tech debt from 2026-09-27 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43538](https://github.com/BerriAI/litellm/pull/43538) - refactor(types): replace Any with proven types in 8 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43551](https://github.com/BerriAI/litellm/pull/43551) - test(rust): enforce shared upstream error contract in wheel checks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43520](https://github.com/BerriAI/litellm/pull/43520) - fix(proxy): log key owner identity on expired key auth failures by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43105](https://github.com/BerriAI/litellm/pull/43105) - fix(cost-map): registry audit 2026-09-28, openai deep-research shutdown dates, azure deepseek v4.1 flash direct price, vertex gemini 3.8 live avatar price, drop azure_ai/muse-spark-1.3 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43566](https://github.com/BerriAI/litellm/pull/43566) - security(proxy): keep team callback credentials out of the stored request body by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43217](https://github.com/BerriAI/litellm/pull/43217) - feat(model_prices): add claude-sonnet-5-5 model pricing by [@krrish-berri-2](https://github.com/krrish-berri-2) in [#43586](https://github.com/BerriAI/litellm/pull/43586) - test(integration): pin org-admin status codes in the team-admin matrix by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43592](https://github.com/BerriAI/litellm/pull/43592) - refactor(guardrails): fix agent 365 to the production endpoint and log the opt-in fail_open at error level by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43189](https://github.com/BerriAI/litellm/pull/43189) - fix(model_prices): correct Claude Sonnet 5.5 capabilities and provider keys by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43587](https://github.com/BerriAI/litellm/pull/43587) - test(unit): stop test modules from putting their own directory on sys.path by [@yuneng-berri](https://github.com/yuneng-berri) in [#43421](https://github.com/BerriAI/litellm/pull/43421) - refactor(rust): centralize host execution and compose callbacks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43515](https://github.com/BerriAI/litellm/pull/43515) - fix(proxy): unregister logging callbacks removed from the stored config by [@yuneng-berri](https://github.com/yuneng-berri) in [#43428](https://github.com/BerriAI/litellm/pull/43428) - chore(cost-map): take azure limits for deepseek-v4-flash-0731 and v3.2-speciale by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43597](https://github.com/BerriAI/litellm/pull/43597) - revert: "feat(proxy): server-side Team Usage export beyond the top-N key cap ([#42996](https://github.com/BerriAI/litellm/issues/42996))" by [@yassin-berriai](https://github.com/yassin-berriai) in [#43376](https://github.com/BerriAI/litellm/pull/43376) - fix(cost-map): add web search flag and model page source to anthropic claude-sonnet-5-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43584](https://github.com/BerriAI/litellm/pull/43584) - refactor(mcp): consolidate hub publication predicate by [@tin-berri](https://github.com/tin-berri) in [#43394](https://github.com/BerriAI/litellm/pull/43394) - fix(provider): accept 2xx status codes in unified_access_group create by [@Louis-Vauterin](https://github.com/Louis-Vauterin) in [#42461](https://github.com/BerriAI/litellm/pull/42461) - fix(panw_prisma_airs): honor experimental_use_latest_role_message_only on every request shape by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42447](https://github.com/BerriAI/litellm/pull/42447) - fix(cost-map): add Vertex batch cache prices to vertex_ai/claude-sonnet-5-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43602](https://github.com/BerriAI/litellm/pull/43602) - revert: "feat(usage): search team keys beyond the top-N in the Team usage view ([#42857](https://github.com/BerriAI/litellm/issues/42857))" by [@yassin-berriai](https://github.com/yassin-berriai) in [#43377](https://github.com/BerriAI/litellm/pull/43377) - feat(providers): add Prism provider (internal copy of [#40914](https://github.com/BerriAI/litellm/issues/40914)) by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41961](https://github.com/BerriAI/litellm/pull/41961) - test(integration): credential canary suite harness by [@yucheng-berri](https://github.com/yucheng-berri) in [#43300](https://github.com/BerriAI/litellm/pull/43300) - feat(bedrock): add xai grok-4.7 pricing and sync llama, mistral large 2407 and minimax m2.5 prices by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43623](https://github.com/BerriAI/litellm/pull/43623) - ci: sync the weekly release cycle with Linear releases by [@yuneng-berri](https://github.com/yuneng-berri) in [#43636](https://github.com/BerriAI/litellm/pull/43636) - feat(cache): select Rust caching through explicit cache objects by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43601](https://github.com/BerriAI/litellm/pull/43601) - test(integration): credential canary slots for MCP and pass-through credentials by [@yucheng-berri](https://github.com/yucheng-berri) in [#43308](https://github.com/BerriAI/litellm/pull/43308) - test(integration): stored-config credential canary slots by [@yucheng-berri](https://github.com/yucheng-berri) in [#43309](https://github.com/BerriAI/litellm/pull/43309) - fix(guardrails): preserve Presidio output selection and restoration by [@joshua-berri](https://github.com/joshua-berri) in [#43401](https://github.com/BerriAI/litellm/pull/43401) - fix(ui): rename All Models tab to Deployed Models and model filters to All Proxy Models by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43638](https://github.com/BerriAI/litellm/pull/43638) - feat(mcp): scan and pin upstream tool descriptions by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43283](https://github.com/BerriAI/litellm/pull/43283) - feat(cost-map): add bedrock_mantle rows for claude opus 5.5 and sonnet 5.5 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43647](https://github.com/BerriAI/litellm/pull/43647) - chore(codeowners): drop UI, migration, and CODEOWNERS self owners by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43653](https://github.com/BerriAI/litellm/pull/43653) - feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43641](https://github.com/BerriAI/litellm/pull/43641) - feat: add model leaderboard page by [@ishaan-berri](https://github.com/ishaan-berri) in [#43649](https://github.com/BerriAI/litellm/pull/43649) - fix(google_genai): preserve proxy_server_request in completion adapter by [@hsm207](https://github.com/hsm207) in [#43536](https://github.com/BerriAI/litellm/pull/43536) - fix(otel): send cache and reasoning tokens in langfuse usage_details by [@ankit373](https://github.com/ankit373) in [#43553](https://github.com/BerriAI/litellm/pull/43553) - fix(vertex_ai): forward the per-turn-control beta for per-message output_config by [@DeviaVir](https://github.com/DeviaVir) in [#43558](https://github.com/BerriAI/litellm/pull/43558) - fix(cost-map): add tool calling and reasoning flags, correct max output for nebius DeepSeek-V4.1-Flash by [@Flexomatic81](https://github.com/Flexomatic81) in [#43588](https://github.com/BerriAI/litellm/pull/43588) - fix(proxy): resolve model_group_alias in the zero-cost budget predicate by [@fedaeho](https://github.com/fedaeho) in [#43512](https://github.com/BerriAI/litellm/pull/43512) - refactor: clean up fresh tech debt from 2026-09-28 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43674](https://github.com/BerriAI/litellm/pull/43674) - refactor(types): replace Any with proven types in 7 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43704](https://github.com/BerriAI/litellm/pull/43704) - docs(security): point readers to the security announcements mailing list signup by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43713](https://github.com/BerriAI/litellm/pull/43713) - refactor(rust): add shared llms wire type derives by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43730](https://github.com/BerriAI/litellm/pull/43730) - fix(router): stream /v1/messages lifecycle frames live when no fallback can take over by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43600](https://github.com/BerriAI/litellm/pull/43600) - feat(cost-map): add baseten DeepSeek-V4.1-Flash-Fast by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43735](https://github.com/BerriAI/litellm/pull/43735) - feat(guardrails): send a configured gateway_name from noma_v2 to Noma by [@itaimodi](https://github.com/itaimodi) in [#43678](https://github.com/BerriAI/litellm/pull/43678) - chore(cost-map): add openai gpt-6.1-sol from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43738](https://github.com/BerriAI/litellm/pull/43738) - test(integration): callback credential canary slots C1-C3 and D5 by [@yucheng-berri](https://github.com/yucheng-berri) in [#43630](https://github.com/BerriAI/litellm/pull/43630) - test(integration): request-path credential canary slots D1-D4 by [@yucheng-berri](https://github.com/yucheng-berri) in [#43307](https://github.com/BerriAI/litellm/pull/43307) - test(integration): sweep proxy logs, metrics, a Datadog intake and the Logs drawer for credential canaries by [@yucheng-berri](https://github.com/yucheng-berri) in [#43306](https://github.com/BerriAI/litellm/pull/43306) - chore(cost-map): add azure and openrouter gpt-6.1-sol rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43744](https://github.com/BerriAI/litellm/pull/43744) - test(proxy): classify every credential-bearing param for the canary suite by [@yucheng-berri](https://github.com/yucheng-berri) in [#43298](https://github.com/BerriAI/litellm/pull/43298) - fix(cost-map): lower fireworks up-to-4b size tier to the pricing page price by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43740](https://github.com/BerriAI/litellm/pull/43740) - fix(cost_calculator): bill chat per-second pricing once with a new cost_per_second field by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43614](https://github.com/BerriAI/litellm/pull/43614) - chore(cost-map): add openai gpt-6-astra ultrafast tier prices from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43745](https://github.com/BerriAI/litellm/pull/43745) - fix(autorouter): compare historical and new savings consistently by [@tin-berri](https://github.com/tin-berri) in [#43348](https://github.com/BerriAI/litellm/pull/43348) - fix(model-prices): align Azure, Bedrock, Copilot, Gemini, Groq, OpenAI and OpenRouter entries with official docs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43598](https://github.com/BerriAI/litellm/pull/43598) - fix(guardrails): enable explicit PANW MCP output scanning by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43109](https://github.com/BerriAI/litellm/pull/43109) - refactor(rust): orchestrate Messages route execution by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43719](https://github.com/BerriAI/litellm/pull/43719) - fix(streaming): keep the served service_tier on streamed chunks and spend rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42870](https://github.com/BerriAI/litellm/pull/42870) - feat(bedrock): add openai gpt-6.1-sol global and base rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43758](https://github.com/BerriAI/litellm/pull/43758) - perf(router): fetch cooldown state and usage counters in one Redis round trip by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43320](https://github.com/BerriAI/litellm/pull/43320) - perf(proxy): hold one spend counter batch across admission and across post-call accounting by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43369](https://github.com/BerriAI/litellm/pull/43369) - perf(responses): run aresponses through the async wrapper so the cache is read once by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43769](https://github.com/BerriAI/litellm/pull/43769) - fix(router): bind Claude Code background sessions to their auto-router by [@tin-berri](https://github.com/tin-berri) in [#43767](https://github.com/BerriAI/litellm/pull/43767) - feat(bedrock): add openai.gpt-6.1-sol us geo cris and Mantle rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43763](https://github.com/BerriAI/litellm/pull/43763) - fix(proxy): recover session key owners from daily spend for usage attribution by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43642](https://github.com/BerriAI/litellm/pull/43642) - chore(cost-map): take azure context limits from models-sold-directly by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43759](https://github.com/BerriAI/litellm/pull/43759) - perf(proxy): one request-scoped Redis pipeline for auth, spend, rate-limit and routing reads by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43407](https://github.com/BerriAI/litellm/pull/43407) - fix(mcp): scope OpenAPI listings to the exact server prefix and drop upstream OAuth metadata when a server is saved by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43608](https://github.com/BerriAI/litellm/pull/43608) - perf(proxy): one post-call Redis pipeline per backend for spend, rate-limit, routing and response-cache writes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43779](https://github.com/BerriAI/litellm/pull/43779) - fix(bedrock): set gpt-6.1-sol max output tokens to 131072 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43782](https://github.com/BerriAI/litellm/pull/43782) - perf(proxy): refresh auth management objects through the request Redis pipeline by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43776](https://github.com/BerriAI/litellm/pull/43776) - feat(agents): add identity storage and validation contracts by [@joshua-berri](https://github.com/joshua-berri) in [#43720](https://github.com/BerriAI/litellm/pull/43720) - chore: bump litellm-enterprise 0.1.71 -> 0.1.72, litellm-proxy-extras 0.4.102 -> 0.4.103, litellm 1.104.0 -> 1.105.0 by [@yuneng-berri](https://github.com/yuneng-berri) in [#43789](https://github.com/BerriAI/litellm/pull/43789) - fix(auth): give UI/CLI session tokens their own AES-GCM context and header-safe shape by [@yuneng-berri](https://github.com/yuneng-berri) in [#43790](https://github.com/BerriAI/litellm/pull/43790) - chore(deps): bump pyjwt, moment and brace-expansion to clear osv-scan by [@yuneng-berri](https://github.com/yuneng-berri) in [#43792](https://github.com/BerriAI/litellm/pull/43792) - fix(ui): surface x-litellm-call-id in Logs search, table and drawer by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42436](https://github.com/BerriAI/litellm/pull/42436) - fix(proxy): look up hashed key names with two spend log rows per key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43656](https://github.com/BerriAI/litellm/pull/43656) - fix(cost-map): add deprecation_date to two together_ai nvidia rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43809](https://github.com/BerriAI/litellm/pull/43809) - test(ci): refresh retired OpenAI tool-call models by [@yuneng-berri](https://github.com/yuneng-berri) in [#43676](https://github.com/BerriAI/litellm/pull/43676) - fix(ui): keep MCP permissions visible after key, team and MCP server saves by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43810](https://github.com/BerriAI/litellm/pull/43810) - test(ci): repair MCP Responses and budget fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43788](https://github.com/BerriAI/litellm/pull/43788) - test(bedrock): accept regional aliases that inherit Converse routing by [@yuneng-berri](https://github.com/yuneng-berri) in [#43785](https://github.com/BerriAI/litellm/pull/43785) - fix(params): categorize internal param appropriately to prevent leaking into request by [@shrey-berri](https://github.com/shrey-berri) in [#43783](https://github.com/BerriAI/litellm/pull/43783) - perf(router): honour the cooldown read interval in the routing prefetch by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43815](https://github.com/BerriAI/litellm/pull/43815) - fix(router): carry per-request routing reads on context variables instead of public method kwargs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43814](https://github.com/BerriAI/litellm/pull/43814) - fix(bedrock): add beta header for output config in message by [@shrey-berri](https://github.com/shrey-berri) in [#43778](https://github.com/BerriAI/litellm/pull/43778) - refactor: clean up fresh tech debt from 2026-09-29 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43830](https://github.com/BerriAI/litellm/pull/43830) - test(router): settle the shared logging worker before recording shadow callbacks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43847](https://github.com/BerriAI/litellm/pull/43847) - chore(model_prices): add Gemini Veo, Mistral and Azure Claude 4.5 deprecation dates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43857](https://github.com/BerriAI/litellm/pull/43857) - fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43764](https://github.com/BerriAI/litellm/pull/43764) - refactor(rust): centralize Python bridge execution wrappers by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43871](https://github.com/BerriAI/litellm/pull/43871) - chore(cost-map): add openai gpt-image-2.5 batch prices from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43869](https://github.com/BerriAI/litellm/pull/43869) - chore(cost-map): add fireworks priority prices for ember-1, nemotron and glm 5.3 us rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43811](https://github.com/BerriAI/litellm/pull/43811) - fix(proxy): attribute completed batch cost rows to /batches in daily activity by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43870](https://github.com/BerriAI/litellm/pull/43870) - fix(router): strip encrypted reasoning the pinned deployment cannot decrypt by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43781](https://github.com/BerriAI/litellm/pull/43781) - fix(cost_calculator): stop copying optional_params into response hidden params by [@yucheng-berri](https://github.com/yucheng-berri) in [#43637](https://github.com/BerriAI/litellm/pull/43637) - feat(pricing): add vertex_ai gemini-3.8 flash tts rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43876](https://github.com/BerriAI/litellm/pull/43876) - fix(proxy): keep request-body credentials out of stored spend-log requests by [@yucheng-berri](https://github.com/yucheng-berri) in [#43635](https://github.com/BerriAI/litellm/pull/43635) - fix(ui): right-align money and count columns across tables by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#37889](https://github.com/BerriAI/litellm/pull/37889) - feat(agents): enforce authoritative agent permissions by [@joshua-berri](https://github.com/joshua-berri) in [#43721](https://github.com/BerriAI/litellm/pull/43721) - fix(proxy): strip caller credentials from websocket passthrough by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43855](https://github.com/BerriAI/litellm/pull/43855) - fix(ui): render access group MCP and agent selections as wrapping chips by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41228](https://github.com/BerriAI/litellm/pull/41228) - fix(responses): scan and mask top-level instructions with guardrails by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43629](https://github.com/BerriAI/litellm/pull/43629) - feat(traces): add Rust storage foundation by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43819](https://github.com/BerriAI/litellm/pull/43819) - feat(ui): agent traces tab on logs with timeline and otel setup guide by [@ishaan-berri](https://github.com/ishaan-berri) in [#43891](https://github.com/BerriAI/litellm/pull/43891) - feat(otel v2): excluded_services opt-out for datastore spans on tenant destinations by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43278](https://github.com/BerriAI/litellm/pull/43278) - fix(traces): correct ClickHouse rollup partitioning, dedupe keys, and retention changes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43901](https://github.com/BerriAI/litellm/pull/43901) - fix(bedrock): keep applicable beta headers by [@shrey-berri](https://github.com/shrey-berri) in [#43829](https://github.com/BerriAI/litellm/pull/43829) - feat(agents): authenticate Entra identities and delegated requests by [@joshua-berri](https://github.com/joshua-berri) in [#43722](https://github.com/BerriAI/litellm/pull/43722) - fix(hosted_vllm): keep reasoning_content on replayed assistant messages by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43599](https://github.com/BerriAI/litellm/pull/43599) - fix(proxy): delete large teams without per-member transaction fan-out by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42998](https://github.com/BerriAI/litellm/pull/42998) - feat(tracing): port OTLP ingestion to current trace foundation by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43915](https://github.com/BerriAI/litellm/pull/43915) - feat(proxy): add native ROI calculator for gateway spend vs merged PRs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43669](https://github.com/BerriAI/litellm/pull/43669) - test(e2e): repair completion, SAIL, and spend-log fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43902](https://github.com/BerriAI/litellm/pull/43902) - test(s3_v2): pin async 5xx retry through the production AsyncHTTPHandler by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43080](https://github.com/BerriAI/litellm/pull/43080) - feat(ui): adopt the new LiteLLM logo and monogram by [@yuneng-berri](https://github.com/yuneng-berri) in [#43913](https://github.com/BerriAI/litellm/pull/43913) - fix(proxy): relay Azure passthrough body model groups through the router by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43896](https://github.com/BerriAI/litellm/pull/43896) - fix(proxy): register a UI-configured arize callback next to otel under OTel v2 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43906](https://github.com/BerriAI/litellm/pull/43906) - fix(transcription): honor base_url alias for Groq Whisper and report it as the api base by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43917](https://github.com/BerriAI/litellm/pull/43917) - feat(tracing): store spend in ClickHouse automatically by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43928](https://github.com/BerriAI/litellm/pull/43928) - fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43903](https://github.com/BerriAI/litellm/pull/43903) - feat(agents): add identity registration and dashboard controls by [@joshua-berri](https://github.com/joshua-berri) in [#43723](https://github.com/BerriAI/litellm/pull/43723) - refactor(proxy): answer every team access check with TeamAccess.allows by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43364](https://github.com/BerriAI/litellm/pull/43364) - test(bedrock): restore the AWS env after a failed live call in the auth tests by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43921](https://github.com/BerriAI/litellm/pull/43921) - feat(lens): analyze agent activity with a separate worker by [@moe-berri](https://github.com/moe-berri) in [#43889](https://github.com/BerriAI/litellm/pull/43889) - fix(router): bill service tiers at catalog rates for custom-priced deployments by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43890](https://github.com/BerriAI/litellm/pull/43890) - test(ci): refresh qualified retired OpenAI fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43938](https://github.com/BerriAI/litellm/pull/43938) - fix(anthropic): forward the dangerous-tool-use beta to Azure AI Foundry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43934](https://github.com/BerriAI/litellm/pull/43934) - test(proxy): scope user_api_key_auth overrides in proxy_server tests by [@yuneng-berri](https://github.com/yuneng-berri) in [#43952](https://github.com/BerriAI/litellm/pull/43952) - fix(wandb): set supports_vision true on GLM-5.3-Flash by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43951](https://github.com/BerriAI/litellm/pull/43951) - chore(cost-map): sync openrouter prices from the models API by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43950](https://github.com/BerriAI/litellm/pull/43950) - fix(grayswan): send request conversation and tool calls to post-call monitor by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43770](https://github.com/BerriAI/litellm/pull/43770) - fix(azure_storage): name Data Lake objects without base64 padding or slashes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43914](https://github.com/BerriAI/litellm/pull/43914) - fix(guardrails): treat an unknown straiker api_version as unset instead of skipping the guardrail by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43956](https://github.com/BerriAI/litellm/pull/43956) - chore(deps): bump gitpython and tornado, extend diskcache osv ignore to Nov 1 by [@yuneng-berri](https://github.com/yuneng-berri) in [#43961](https://github.com/BerriAI/litellm/pull/43961) - fix(azure_storage): keep the DataLakeServiceClient alive until its TTL elapses by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43082](https://github.com/BerriAI/litellm/pull/43082) - feat(proxy): record in spend logs whether a request used a client-forwarded Anthropic OAuth token by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43063](https://github.com/BerriAI/litellm/pull/43063) - test(ci): repair stale tests and move retired OpenAI text-completion fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43958](https://github.com/BerriAI/litellm/pull/43958) - feat(e2e): record each e2e test's steps, starting with ProxyClient by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42393](https://github.com/BerriAI/litellm/pull/42393) - feat(providers): add Cortecs as an OpenAI-compatible provider by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43872](https://github.com/BerriAI/litellm/pull/43872) - feat(ui): filter tags by name and description on the Tag Management page by [@galovics](https://github.com/galovics) in [#42949](https://github.com/BerriAI/litellm/pull/42949) - fix(caching): write the response-cache SET to Redis at once instead of on the post-call batch by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43973](https://github.com/BerriAI/litellm/pull/43973) - test(e2e): typed per-test metadata for the e2e suite by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42044](https://github.com/BerriAI/litellm/pull/42044) - feat(guardrails): honor litellm_params.timeout in every HTTP guardrail by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43134](https://github.com/BerriAI/litellm/pull/43134) - chore(cost-map): add fireworks inkling priority prices from the prices api by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43949](https://github.com/BerriAI/litellm/pull/43949) - chore(cost-map): add deprecation date for anthropic claude-sonnet-4-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43898](https://github.com/BerriAI/litellm/pull/43898) - fix(cost-map): raise baseten DeepSeek-V4.1-Flash max output to 262144 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43916](https://github.com/BerriAI/litellm/pull/43916) - fix(proxy): restore pre-config-wins handling of pass-through endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#43962](https://github.com/BerriAI/litellm/pull/43962) - feat(lens): investigate sampled traces and retain batch results by [@moe-berri](https://github.com/moe-berri) in [#43942](https://github.com/BerriAI/litellm/pull/43942) - fix(guardrails): scan Responses API input in Azure Prompt Shield by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43786](https://github.com/BerriAI/litellm/pull/43786) - fix(guardrails): scan Responses API input in Azure Text Moderation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43965](https://github.com/BerriAI/litellm/pull/43965) - refactor: clean up fresh tech debt from 2026-09-30 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43993](https://github.com/BerriAI/litellm/pull/43993) - test: inject the HIBP client and the MCP loop clock so two backend tests stop flaking by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#44007](https://github.com/BerriAI/litellm/pull/44007) - fix(cost-map): reprice fireworks deepseek v4.1 flash to the 2026-10-01 pricing up…

…105.0) (#2711) 🔵 **minor** · ghcr.io/berriai/litellm This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.104.2` → `v1.105.0` | --- > ⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/107) for more information. > ⛔ **Do not automerge.** v1.99.0 silently flipped the MCP `allow_all_keys` default and emptied every tool list. --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.105.0`](https://github.com/BerriAI/litellm/releases/tag/v1.105.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.104.3...v1.105.0) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.105.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.105.0/cosign.pub \ ghcr.io/berriai/litellm:v1.105.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - feat(mcp): share compatibility-aware result conversion across tool surfaces by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43089](https://github.com/BerriAI/litellm/pull/43089) - fix(caching): propagate auth cache invalidation over Redis Cluster via a node-level pub/sub client by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43110](https://github.com/BerriAI/litellm/pull/43110) - feat(model-catalog): add Rust registry validation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43136](https://github.com/BerriAI/litellm/pull/43136) - test(rust): reorganize core crate tests and split cache and OCR suites by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43177](https://github.com/BerriAI/litellm/pull/43177) - fix(proxy): give SpendLogToolIndex its own share of the cleanup budget and log a per-run summary by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41768](https://github.com/BerriAI/litellm/pull/41768) - feat(rust): hand upstream response headers to the native Messages stream by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43178](https://github.com/BerriAI/litellm/pull/43178) - feat(testkit): agent clients for Claude Code, Codex and opencode by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43181](https://github.com/BerriAI/litellm/pull/43181) - ci: run migrated unit selections on every event in legacy GHA shards by [@yuneng-berri](https://github.com/yuneng-berri) in [#43182](https://github.com/BerriAI/litellm/pull/43182) - test: move tests/test_litellm root and small trees into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43186](https://github.com/BerriAI/litellm/pull/43186) - fix(sso): gate /sso/debug routes behind ENABLE_SSO_DEBUG, off by default by [@ojensen-berri](https://github.com/ojensen-berri) in [#43150](https://github.com/BerriAI/litellm/pull/43150) - refactor(framer): replace Framer trait with tokio-util codecs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43193](https://github.com/BerriAI/litellm/pull/43193) - fix(anthropic): surface Responses bridge stream failures as Anthropic error events by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43126](https://github.com/BerriAI/litellm/pull/43126) - test: move tests/test_litellm/llms into tests/unit/llms by [@yuneng-berri](https://github.com/yuneng-berri) in [#43191](https://github.com/BerriAI/litellm/pull/43191) - test: move tests/test_litellm integrations and secret_managers into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43194](https://github.com/BerriAI/litellm/pull/43194) - feat(mcp): configure protocol versions and capability discovery by [@joshua-berri](https://github.com/joshua-berri) in [#43169](https://github.com/BerriAI/litellm/pull/43169) - fix(proxy): record streamed /v1/responses container ownership before the response.completed frame by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43140](https://github.com/BerriAI/litellm/pull/43140) - fix(tests): drop the repeated UNIT_FLAG key in test_unit_shard_missing_paths by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43212](https://github.com/BerriAI/litellm/pull/43212) - fix(router): count provider budget spend on every API surface by [@daqiangganjun](https://github.com/daqiangganjun) in [#38172](https://github.com/BerriAI/litellm/pull/38172) - fix(responses): fall back on pre-output stream drops, fail truncated streams, honor request_timeout by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43133](https://github.com/BerriAI/litellm/pull/43133) - fix(sentry): scrub PII and secrets inside object reprs and nested locals, add SENTRY_SEND_DEFAULT_PII opt-in by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43123](https://github.com/BerriAI/litellm/pull/43123) - fix(bedrock): surface a converse-stream 200 that decodes to no events as a 502 instead of an empty turn by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43213](https://github.com/BerriAI/litellm/pull/43213) - fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43206](https://github.com/BerriAI/litellm/pull/43206) - feat(integrations): add Databricks Zerobus trace logging callback by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42013](https://github.com/BerriAI/litellm/pull/42013) - fix(vertex_ai): keep legacy bucket_name in credential resolution and add GCS_BATCH_BUCKET_NAME env var by [@mubashir1osmani](https://github.com/mubashir1osmani) in [#42803](https://github.com/BerriAI/litellm/pull/42803) - feat(xai): add native xAI batches and files support by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42812](https://github.com/BerriAI/litellm/pull/42812) - fix(proxy): send a real error event when a /v1/messages stream fails by [@4refael](https://github.com/4refael) in [#41826](https://github.com/BerriAI/litellm/pull/41826) - fix(logging): redact raw_request when turn_off_message_logging is set in the proxy config by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43219](https://github.com/BerriAI/litellm/pull/43219) - fix(router): parse the classifier verdict out of surrounding prose instead of falling to the default tier by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43215](https://github.com/BerriAI/litellm/pull/43215) - test(zerobus): move tests into active CI selection by [@tin-berri](https://github.com/tin-berri) in [#43235](https://github.com/BerriAI/litellm/pull/43235) - fix(proxy): keep the submitted body out of 422 validation errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43231](https://github.com/BerriAI/litellm/pull/43231) - test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43199](https://github.com/BerriAI/litellm/pull/43199) - chore(cost-map): add fireworks us-only deepseek v4.1 flash priority prices by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43247](https://github.com/BerriAI/litellm/pull/43247) - chore(cost-map): sync openrouter prices and add perceptron-mk1.5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43246](https://github.com/BerriAI/litellm/pull/43246) - fix(cost-map): correct fireworks_ai deepseek-v4p1-flash pricing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43253](https://github.com/BerriAI/litellm/pull/43253) - feat(openrouter): add typesafe/jev-router to the cost map by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43248](https://github.com/BerriAI/litellm/pull/43248) - chore(cost-map): add fireworks priority prices for muse glimmer 30b and deepseek v4 flash vision exp by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43252](https://github.com/BerriAI/litellm/pull/43252) - refactor(rust): move credential inheritance and SDK limits into a driver preflight by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43259](https://github.com/BerriAI/litellm/pull/43259) - fix(router): hold Responses lifecycle events until output so a pre-output fallback announces one response by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43238](https://github.com/BerriAI/litellm/pull/43238) - refactor(http): hand out an owned Client and route all providers through the pool by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43245](https://github.com/BerriAI/litellm/pull/43245) - test: fix stale and state-leaking tests red on scheduled CircleCI by [@yuneng-berri](https://github.com/yuneng-berri) in [#43266](https://github.com/BerriAI/litellm/pull/43266) - fix(cost-map): retirement dates, chatgpt reasoning flags, bing pricing, bedrock mantle and mythos, azure gpt-5.6 alias, anthropic batch rates, new nebius, openrouter and xai rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42951](https://github.com/BerriAI/litellm/pull/42951) - refactor(rust): promote anthropic messages out of experimental_pass_through by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43269](https://github.com/BerriAI/litellm/pull/43269) - test: stop CI tests from downloading tokenizer files and images by [@yuneng-berri](https://github.com/yuneng-berri) in [#43257](https://github.com/BerriAI/litellm/pull/43257) - fix(rust): preserve nested optional import failures by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43265](https://github.com/BerriAI/litellm/pull/43265) - ci: drop main and litellm\_\* branch filters from the CircleCI litellm-main workflows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43272](https://github.com/BerriAI/litellm/pull/43272) - feat(proxy): email alerts at configured percentages of a team member budget by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42665](https://github.com/BerriAI/litellm/pull/42665) - fix(callbacks-legacy-python): traverse and release the retained headers dict by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43274](https://github.com/BerriAI/litellm/pull/43274) - test(integration): port langfuse callbacks-in-db coverage to the local harness by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43282](https://github.com/BerriAI/litellm/pull/43282) - test(e2e): accept the otel cost write as a linked root trace by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42931](https://github.com/BerriAI/litellm/pull/42931) - test: finish the non-proxy half of tests/test_litellm by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43281](https://github.com/BerriAI/litellm/pull/43281) - refactor(rust): prepare inference and auth foundations for the gateway by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43287](https://github.com/BerriAI/litellm/pull/43287) - feat(rust): add config, router and gateway crates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43289](https://github.com/BerriAI/litellm/pull/43289) - fix(cost-map): remove duplicate openrouter/perceptron/perceptron-mk1.5 entry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43273](https://github.com/BerriAI/litellm/pull/43273) - fix(e2e): resolve the blank-S3 gateway repo root from the litellm package location by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42911](https://github.com/BerriAI/litellm/pull/42911) - test(integration): run the Langfuse DB-callback test on its own scratch database by [@yuneng-berri](https://github.com/yuneng-berri) in [#43288](https://github.com/BerriAI/litellm/pull/43288) - refactor(rust): expand logging and test coverage across gateway and Anthropic messages by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43295](https://github.com/BerriAI/litellm/pull/43295) - refactor: daily fresh tech debt cleanup, rolling PR (2026-09-25) by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43151](https://github.com/BerriAI/litellm/pull/43151) - test(proxy_behavior): scope the management proxy fixture to its package so its spend monitor cannot race the spend tests by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43302](https://github.com/BerriAI/litellm/pull/43302) - fix(cost-map): registry audit 2026-09-26, MAI-Image-2.5-Flash price, Databricks Claude Opus 5.5, Azure Foundry retirement dates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43254](https://github.com/BerriAI/litellm/pull/43254) - fix(cost-map): price fireworks deepseek v4.1 flash at the prices api value by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43311](https://github.com/BerriAI/litellm/pull/43311) - fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts by [@yuneng-berri](https://github.com/yuneng-berri) in [#43294](https://github.com/BerriAI/litellm/pull/43294) - fix(otel): detach post-response service spans by request phase, name redis spans by operation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43237](https://github.com/BerriAI/litellm/pull/43237) - test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test by [@yuneng-berri](https://github.com/yuneng-berri) in [#43321](https://github.com/BerriAI/litellm/pull/43321) - test(integration): pin the team-admin status-code matrix across every management route by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43249](https://github.com/BerriAI/litellm/pull/43249) - feat(proxy): add fail_closed_rate_limit_enforcement to reject requests with 503 while Redis rate limit counters are unreachable by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43251](https://github.com/BerriAI/litellm/pull/43251) - fix(responses): run stream failure and success hooks on the iterating loop instead of blocking it by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43270](https://github.com/BerriAI/litellm/pull/43270) - fix(guardrails): block private destinations in custom code http_request and bound guardrail execution time by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43280](https://github.com/BerriAI/litellm/pull/43280) - feat(sail): add Sail as a provider with service_tier mapped to its completion window by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42840](https://github.com/BerriAI/litellm/pull/42840) - test(mcp): pin server resolution and authorization behavior by [@joshua-berri](https://github.com/joshua-berri) in [#43261](https://github.com/BerriAI/litellm/pull/43261) - refactor(anthropic): rename experimental_pass_through to pass_through by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43329](https://github.com/BerriAI/litellm/pull/43329) - refactor(mcp): add shared server resolver without changing callers by [@joshua-berri](https://github.com/joshua-berri) in [#43262](https://github.com/BerriAI/litellm/pull/43262) - ci: fail on new unbounded SQL IN lists and add a Prisma chunking helper by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42629](https://github.com/BerriAI/litellm/pull/42629) - test(logging): drain the logging worker after each logging callback test so no later test inherits its events by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43344](https://github.com/BerriAI/litellm/pull/43344) - feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43271](https://github.com/BerriAI/litellm/pull/43271) - fix(mcp): adopt shared server resolution and caller authorization by [@joshua-berri](https://github.com/joshua-berri) in [#43263](https://github.com/BerriAI/litellm/pull/43263) - fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43022](https://github.com/BerriAI/litellm/pull/43022) - feat(proxy): add maximum_daily_tag_spend_retention_period cleanup setting by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#39221](https://github.com/BerriAI/litellm/pull/39221) - fix(cost-map): sync OpenRouter, Together, Cohere and Azure AI registry values with official sources by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43337](https://github.com/BerriAI/litellm/pull/43337) - docs(pr-template): add the backport-stable label only for a P0 regression by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43351](https://github.com/BerriAI/litellm/pull/43351) - fix(params): keep *litellm*\* kwargs out of provider request bodies by construction by [@shrey-berri](https://github.com/shrey-berri) in [#43221](https://github.com/BerriAI/litellm/pull/43221) - test: remove substring guard test_default_api_base by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43355](https://github.com/BerriAI/litellm/pull/43355) - ci: cut CircleCI wall time without loosening test isolation by [@yuneng-berri](https://github.com/yuneng-berri) in [#43347](https://github.com/BerriAI/litellm/pull/43347) - fix(cost-map): correct azure gpt-4o-mini tts, transcribe, alias and MAI-Image-2.5 prices by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43357](https://github.com/BerriAI/litellm/pull/43357) - fix(tests): stop VCR recording and replaying a test's own localhost upstream by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43346](https://github.com/BerriAI/litellm/pull/43346) - test(integration): group /v1/messages contracts under tests/integration/messages_endpoint by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43352](https://github.com/BerriAI/litellm/pull/43352) - fix(params): validate stream_chunk_size once, before any provider call by [@shrey-berri](https://github.com/shrey-berri) in [#43222](https://github.com/BerriAI/litellm/pull/43222) - fix(openai): exclude fine-tuned and custom gpt-5-chat aliases from gpt-5 reasoning path by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43185](https://github.com/BerriAI/litellm/pull/43185) - fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43322](https://github.com/BerriAI/litellm/pull/43322) - fix(caching): stand default cache points down when extra_body hides a direct client mark by [@yuneng-berri](https://github.com/yuneng-berri) in [#43341](https://github.com/BerriAI/litellm/pull/43341) - fix(mcp): report reachability without stored credentials by [@tin-berri](https://github.com/tin-berri) in [#43240](https://github.com/BerriAI/litellm/pull/43240) - fix(mcp): align hub publication status and controls by [@tin-berri](https://github.com/tin-berri) in [#43241](https://github.com/BerriAI/litellm/pull/43241) - chore(cost-map): sync openrouter prices for deepseek, minimax, qwen and glm rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43384](https://github.com/BerriAI/litellm/pull/43384) - chore(cost-map): drop stale cache hit field from openrouter deepseek-v4-pro-0813 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43389](https://github.com/BerriAI/litellm/pull/43389) - fix(responses): stream guardrail pre-call block as SSE with a typed output item by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42507](https://github.com/BerriAI/litellm/pull/42507) - feat(e2e): read management routes back from the control plane replicas by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43373](https://github.com/BerriAI/litellm/pull/43373) - feat(rust): add the openai_like chat config foundation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43379](https://github.com/BerriAI/litellm/pull/43379) - feat(router): opt in to prompt-cache cost routing by [@tin-berri](https://github.com/tin-berri) in [#43232](https://github.com/BerriAI/litellm/pull/43232) - feat(otel): add SigNoz preset for OpenTelemetry v2 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43296](https://github.com/BerriAI/litellm/pull/43296) - feat(cli): reuse saved agent setup and add reconfigure by [@tin-berri](https://github.com/tin-berri) in [#43392](https://github.com/BerriAI/litellm/pull/43392) - fix(streaming): keep litellm Usage on text-completion usage chunks by [@yuneng-berri](https://github.com/yuneng-berri) in [#43047](https://github.com/BerriAI/litellm/pull/43047) - fix(vertex_ai): consider tools when validating context caching min tokens by [@agustin18](https://github.com/agustin18) in [#43319](https://github.com/BerriAI/litellm/pull/43319) - fix(anthropic): drop thinking blocks with empty thinking text, not just missing signature by [@shoemoney](https://github.com/shoemoney) in [#38049](https://github.com/BerriAI/litellm/pull/38049) - test(e2e): report batch cleanup leftovers as a plain UserWarning by [@yuneng-berri](https://github.com/yuneng-berri) in [#43405](https://github.com/BerriAI/litellm/pull/43405) - fix(tools): salvage concatenated JSON tool call arguments by [@kumarpriyanshu09](https://github.com/kumarpriyanshu09) in [#43260](https://github.com/BerriAI/litellm/pull/43260) - fix(gemini): forward seed to the Gemini API instead of rejecting it by [@YaseenBashaT](https://github.com/YaseenBashaT) in [#43197](https://github.com/BerriAI/litellm/pull/43197) - fix(vertex_ai): make Gemma fake streams work with traced Responses by [@stewartpark](https://github.com/stewartpark) in [#43147](https://github.com/BerriAI/litellm/pull/43147) - fix(responses): emit the reasoning item on streaming /v1/responses for signature-only thinking by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43414](https://github.com/BerriAI/litellm/pull/43414) - fix(anthropic): forward the per-turn-control beta to Azure AI Foundry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43415](https://github.com/BerriAI/litellm/pull/43415) - test(vertex_ai): assert the usage the Gemma responses stream actually reports by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43422](https://github.com/BerriAI/litellm/pull/43422) - fix(bedrock): keep the provider status code on unprocessable image errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43416](https://github.com/BerriAI/litellm/pull/43416) - fix(vertex_ai): stop importing the vertexai SDK in partner-model completion by [@anmolg1997](https://github.com/anmolg1997) in [#42274](https://github.com/BerriAI/litellm/pull/42274) - fix(token_counter): count Gemini function_declarations tools by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43417](https://github.com/BerriAI/litellm/pull/43417) - refactor(types): replace Any with proven types in 5 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43304](https://github.com/BerriAI/litellm/pull/43304) - chore(cost-map): update azure_ai/grok-4.6 input price from Azure pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43440](https://github.com/BerriAI/litellm/pull/43440) - chore(cost-map): add azure_ai/MAI-Cyber-1-Flash by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43446](https://github.com/BerriAI/litellm/pull/43446) - refactor(rust): share anthropic types, request helpers, and streaming contracts across crates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43426](https://github.com/BerriAI/litellm/pull/43426) - feat(rust): expand gateway configuration parsing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43460](https://github.com/BerriAI/litellm/pull/43460) - refactor(rust): share call lifecycle across route-owned inference by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43461](https://github.com/BerriAI/litellm/pull/43461) - feat(rust): add the HTTP host driver by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43462](https://github.com/BerriAI/litellm/pull/43462) - build(rust): package the gateway container by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43471](https://github.com/BerriAI/litellm/pull/43471) - refactor(rust): use shared execution in gateway inference by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43463](https://github.com/BerriAI/litellm/pull/43463) - fix(cost-map): add deprecation_date to together_ai Salesforce/Llama-Rank-V1 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43507](https://github.com/BerriAI/litellm/pull/43507) - feat(rust): support the HTTP Responses API by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43464](https://github.com/BerriAI/litellm/pull/43464) - fix(cost-map): sync openrouter prices from the models API by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43506](https://github.com/BerriAI/litellm/pull/43506) - fix(cost-map): set together_ai gpt-oss-20b and gemma-4-31B-it deprecation_date to 2026-09-15 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43509](https://github.com/BerriAI/litellm/pull/43509) - feat(rust): add litellm-db and litellm-db-testing workspace scaffolding by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43504](https://github.com/BerriAI/litellm/pull/43504) - feat(rust): connect Python inference bindings to shared routes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43465](https://github.com/BerriAI/litellm/pull/43465) - feat(rust): add structured route lifecycle tracing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43466](https://github.com/BerriAI/litellm/pull/43466) - feat(rust): separate gateway authentication and authorization by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43467](https://github.com/BerriAI/litellm/pull/43467) - feat(rust): add virtual key storage contracts by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43468](https://github.com/BerriAI/litellm/pull/43468) - feat(rust): add gateway UI login and sessions by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43469](https://github.com/BerriAI/litellm/pull/43469) - feat(rust): add the MCP gateway by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43470](https://github.com/BerriAI/litellm/pull/43470) - refactor(rust): remove delivery routing abstraction by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43514](https://github.com/BerriAI/litellm/pull/43514) - feat(azure): add mistral ocr pricing and azure max output limits by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43530](https://github.com/BerriAI/litellm/pull/43530) - refactor: clean up fresh tech debt from 2026-09-27 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43538](https://github.com/BerriAI/litellm/pull/43538) - refactor(types): replace Any with proven types in 8 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43551](https://github.com/BerriAI/litellm/pull/43551) - test(rust): enforce shared upstream error contract in wheel checks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43520](https://github.com/BerriAI/litellm/pull/43520) - fix(proxy): log key owner identity on expired key auth failures by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43105](https://github.com/BerriAI/litellm/pull/43105) - fix(cost-map): registry audit 2026-09-28, openai deep-research shutdown dates, azure deepseek v4.1 flash direct price, vertex gemini 3.8 live avatar price, drop azure_ai/muse-spark-1.3 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43566](https://github.com/BerriAI/litellm/pull/43566) - security(proxy): keep team callback credentials out of the stored request body by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43217](https://github.com/BerriAI/litellm/pull/43217) - feat(model_prices): add claude-sonnet-5-5 model pricing by [@krrish-berri-2](https://github.com/krrish-berri-2) in [#43586](https://github.com/BerriAI/litellm/pull/43586) - test(integration): pin org-admin status codes in the team-admin matrix by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43592](https://github.com/BerriAI/litellm/pull/43592) - refactor(guardrails): fix agent 365 to the production endpoint and log the opt-in fail_open at error level by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43189](https://github.com/BerriAI/litellm/pull/43189) - fix(model_prices): correct Claude Sonnet 5.5 capabilities and provider keys by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43587](https://github.com/BerriAI/litellm/pull/43587) - test(unit): stop test modules from putting their own directory on sys.path by [@yuneng-berri](https://github.com/yuneng-berri) in [#43421](https://github.com/BerriAI/litellm/pull/43421) - refactor(rust): centralize host execution and compose callbacks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43515](https://github.com/BerriAI/litellm/pull/43515) - fix(proxy): unregister logging callbacks removed from the stored config by [@yuneng-berri](https://github.com/yuneng-berri) in [#43428](https://github.com/BerriAI/litellm/pull/43428) - chore(cost-map): take azure limits for deepseek-v4-flash-0731 and v3.2-speciale by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43597](https://github.com/BerriAI/litellm/pull/43597) - revert: "feat(proxy): server-side Team Usage export beyond the top-N key cap ([#42996](https://github.com/BerriAI/litellm/issues/42996))" by [@yassin-berriai](https://github.com/yassin-berriai) in [#43376](https://github.com/BerriAI/litellm/pull/43376) - fix(cost-map): add web search flag and model page source to anthropic claude-sonnet-5-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43584](https://github.com/BerriAI/litellm/pull/43584) - refactor(mcp): consolidate hub publication predicate by [@tin-berri](https://github.com/tin-berri) in [#43394](https://github.com/BerriAI/litellm/pull/43394) - fix(provider): accept 2xx status codes in unified_access_group create by [@Louis-Vauterin](https://github.com/Louis-Vauterin) in [#42461](https://github.com/BerriAI/litellm/pull/42461) - fix(panw_prisma_airs): honor experimental_use_latest_role_message_only on every request shape by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42447](https://github.com/BerriAI/litellm/pull/42447) - fix(cost-map): add Vertex batch cache prices to vertex_ai/claude-sonnet-5-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43602](https://github.com/BerriAI/litellm/pull/43602) - revert: "feat(usage): search team keys beyond the top-N in the Team usage view ([#42857](https://github.com/BerriAI/litellm/issues/42857))" by [@yassin-berriai](https://github.com/yassin-berriai) in [#43377](https://github.com/BerriAI/litellm/pull/43377) - feat(providers): add Prism provider (internal copy of [#40914](https://github.com/BerriAI/litellm/issues/40914)) by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41961](https://github.com/BerriAI/litellm/pull/41961) - test(integration): credential canary suite harness by [@yucheng-berri](https://github.com/yucheng-berri) in [#43300](https://github.com/BerriAI/litellm/pull/43300) - feat(bedrock): add xai grok-4.7 pricing and sync llama, mistral large 2407 and minimax m2.5 prices by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43623](https://github.com/BerriAI/litellm/pull/43623) - ci: sync the weekly release cycle with Linear releases by [@yuneng-berri](https://github.com/yuneng-berri) in [#43636](https://github.com/BerriAI/litellm/pull/43636) - feat(cache): select Rust caching through explicit cache objects by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43601](https://github.com/BerriAI/litellm/pull/43601) - test(integration): credential canary slots for MCP and pass-through credentials by [@yucheng-berri](https://github.com/yucheng-berri) in [#43308](https://github.com/BerriAI/litellm/pull/43308) - test(integration): stored-config credential canary slots by [@yucheng-berri](https://github.com/yucheng-berri) in [#43309](https://github.com/BerriAI/litellm/pull/43309) - fix(guardrails): preserve Presidio output selection and restoration by [@joshua-berri](https://github.com/joshua-berri) in [#43401](https://github.com/BerriAI/litellm/pull/43401) - fix(ui): rename All Models tab to Deployed Models and model filters to All Proxy Models by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43638](https://github.com/BerriAI/litellm/pull/43638) - feat(mcp): scan and pin upstream tool descriptions by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43283](https://github.com/BerriAI/litellm/pull/43283) - feat(cost-map): add bedrock_mantle rows for claude opus 5.5 and sonnet 5.5 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43647](https://github.com/BerriAI/litellm/pull/43647) - chore(codeowners): drop UI, migration, and CODEOWNERS self owners by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43653](https://github.com/BerriAI/litellm/pull/43653) - feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43641](https://github.com/BerriAI/litellm/pull/43641) - feat: add model leaderboard page by [@ishaan-berri](https://github.com/ishaan-berri) in [#43649](https://github.com/BerriAI/litellm/pull/43649) - fix(google_genai): preserve proxy_server_request in completion adapter by [@hsm207](https://github.com/hsm207) in [#43536](https://github.com/BerriAI/litellm/pull/43536) - fix(otel): send cache and reasoning tokens in langfuse usage_details by [@ankit373](https://github.com/ankit373) in [#43553](https://github.com/BerriAI/litellm/pull/43553) - fix(vertex_ai): forward the per-turn-control beta for per-message output_config by [@DeviaVir](https://github.com/DeviaVir) in [#43558](https://github.com/BerriAI/litellm/pull/43558) - fix(cost-map): add tool calling and reasoning flags, correct max output for nebius DeepSeek-V4.1-Flash by [@Flexomatic81](https://github.com/Flexomatic81) in [#43588](https://github.com/BerriAI/litellm/pull/43588) - fix(proxy): resolve model_group_alias in the zero-cost budget predicate by [@fedaeho](https://github.com/fedaeho) in [#43512](https://github.com/BerriAI/litellm/pull/43512) - refactor: clean up fresh tech debt from 2026-09-28 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43674](https://github.com/BerriAI/litellm/pull/43674) - refactor(types): replace Any with proven types in 7 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43704](https://github.com/BerriAI/litellm/pull/43704) - docs(security): point readers to the security announcements mailing list signup by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43713](https://github.com/BerriAI/litellm/pull/43713) - refactor(rust): add shared llms wire type derives by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43730](https://github.com/BerriAI/litellm/pull/43730) - fix(router): stream /v1/messages lifecycle frames live when no fallback can take over by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43600](https://github.com/BerriAI/litellm/pull/43600) - feat(cost-map): add baseten DeepSeek-V4.1-Flash-Fast by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43735](https://github.com/BerriAI/litellm/pull/43735) - feat(guardrails): send a configured gateway_name from noma_v2 to Noma by [@itaimodi](https://github.com/itaimodi) in [#43678](https://github.com/BerriAI/litellm/pull/43678) - chore(cost-map): add openai gpt-6.1-sol from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43738](https://github.com/BerriAI/litellm/pull/43738) - test(integration): callback credential canary slots C1-C3 and D5 by [@yucheng-berri](https://github.com/yucheng-berri) in [#43630](https://github.com/BerriAI/litellm/pull/43630) - test(integration): request-path credential canary slots D1-D4 by [@yucheng-berri](https://github.com/yucheng-berri) in [#43307](https://github.com/BerriAI/litellm/pull/43307) - test(integration): sweep proxy logs, metrics, a Datadog intake and the Logs drawer for credential canaries by [@yucheng-berri](https://github.com/yucheng-berri) in [#43306](https://github.com/BerriAI/litellm/pull/43306) - chore(cost-map): add azure and openrouter gpt-6.1-sol rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43744](https://github.com/BerriAI/litellm/pull/43744) - test(proxy): classify every credential-bearing param for the canary suite by [@yucheng-berri](https://github.com/yucheng-berri) in [#43298](https://github.com/BerriAI/litellm/pull/43298) - fix(cost-map): lower fireworks up-to-4b size tier to the pricing page price by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43740](https://github.com/BerriAI/litellm/pull/43740) - fix(cost_calculator): bill chat per-second pricing once with a new cost_per_second field by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43614](https://github.com/BerriAI/litellm/pull/43614) - chore(cost-map): add openai gpt-6-astra ultrafast tier prices from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43745](https://github.com/BerriAI/litellm/pull/43745) - fix(autorouter): compare historical and new savings consistently by [@tin-berri](https://github.com/tin-berri) in [#43348](https://github.com/BerriAI/litellm/pull/43348) - fix(model-prices): align Azure, Bedrock, Copilot, Gemini, Groq, OpenAI and OpenRouter entries with official docs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43598](https://github.com/BerriAI/litellm/pull/43598) - fix(guardrails): enable explicit PANW MCP output scanning by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43109](https://github.com/BerriAI/litellm/pull/43109) - refactor(rust): orchestrate Messages route execution by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43719](https://github.com/BerriAI/litellm/pull/43719) - fix(streaming): keep the served service_tier on streamed chunks and spend rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42870](https://github.com/BerriAI/litellm/pull/42870) - feat(bedrock): add openai gpt-6.1-sol global and base rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43758](https://github.com/BerriAI/litellm/pull/43758) - perf(router): fetch cooldown state and usage counters in one Redis round trip by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43320](https://github.com/BerriAI/litellm/pull/43320) - perf(proxy): hold one spend counter batch across admission and across post-call accounting by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43369](https://github.com/BerriAI/litellm/pull/43369) - perf(responses): run aresponses through the async wrapper so the cache is read once by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43769](https://github.com/BerriAI/litellm/pull/43769) - fix(router): bind Claude Code background sessions to their auto-router by [@tin-berri](https://github.com/tin-berri) in [#43767](https://github.com/BerriAI/litellm/pull/43767) - feat(bedrock): add openai.gpt-6.1-sol us geo cris and Mantle rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43763](https://github.com/BerriAI/litellm/pull/43763) - fix(proxy): recover session key owners from daily spend for usage attribution by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43642](https://github.com/BerriAI/litellm/pull/43642) - chore(cost-map): take azure context limits from models-sold-directly by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43759](https://github.com/BerriAI/litellm/pull/43759) - perf(proxy): one request-scoped Redis pipeline for auth, spend, rate-limit and routing reads by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43407](https://github.com/BerriAI/litellm/pull/43407) - fix(mcp): scope OpenAPI listings to the exact server prefix and drop upstream OAuth metadata when a server is saved by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43608](https://github.com/BerriAI/litellm/pull/43608) - perf(proxy): one post-call Redis pipeline per backend for spend, rate-limit, routing and response-cache writes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43779](https://github.com/BerriAI/litellm/pull/43779) - fix(bedrock): set gpt-6.1-sol max output tokens to 131072 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43782](https://github.com/BerriAI/litellm/pull/43782) - perf(proxy): refresh auth management objects through the request Redis pipeline by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43776](https://github.com/BerriAI/litellm/pull/43776) - feat(agents): add identity storage and validation contracts by [@joshua-berri](https://github.com/joshua-berri) in [#43720](https://github.com/BerriAI/litellm/pull/43720) - chore: bump litellm-enterprise 0.1.71 -> 0.1.72, litellm-proxy-extras 0.4.102 -> 0.4.103, litellm 1.104.0 -> 1.105.0 by [@yuneng-berri](https://github.com/yuneng-berri) in [#43789](https://github.com/BerriAI/litellm/pull/43789) - fix(auth): give UI/CLI session tokens their own AES-GCM context and header-safe shape by [@yuneng-berri](https://github.com/yuneng-berri) in [#43790](https://github.com/BerriAI/litellm/pull/43790) - chore(deps): bump pyjwt, moment and brace-expansion to clear osv-scan by [@yuneng-berri](https://github.com/yuneng-berri) in [#43792](https://github.com/BerriAI/litellm/pull/43792) - fix(ui): surface x-litellm-call-id in Logs search, table and drawer by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42436](https://github.com/BerriAI/litellm/pull/42436) - fix(proxy): look up hashed key names with two spend log rows per key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43656](https://github.com/BerriAI/litellm/pull/43656) - fix(cost-map): add deprecation_date to two together_ai nvidia rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43809](https://github.com/BerriAI/litellm/pull/43809) - test(ci): refresh retired OpenAI tool-call models by [@yuneng-berri](https://github.com/yuneng-berri) in [#43676](https://github.com/BerriAI/litellm/pull/43676) - fix(ui): keep MCP permissions visible after key, team and MCP server saves by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43810](https://github.com/BerriAI/litellm/pull/43810) - test(ci): repair MCP Responses and budget fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43788](https://github.com/BerriAI/litellm/pull/43788) - test(bedrock): accept regional aliases that inherit Converse routing by [@yuneng-berri](https://github.com/yuneng-berri) in [#43785](https://github.com/BerriAI/litellm/pull/43785) - fix(params): categorize internal param appropriately to prevent leaking into request by [@shrey-berri](https://github.com/shrey-berri) in [#43783](https://github.com/BerriAI/litellm/pull/43783) - perf(router): honour the cooldown read interval in the routing prefetch by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43815](https://github.com/BerriAI/litellm/pull/43815) - fix(router): carry per-request routing reads on context variables instead of public method kwargs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43814](https://github.com/BerriAI/litellm/pull/43814) - fix(bedrock): add beta header for output config in message by [@shrey-berri](https://github.com/shrey-berri) in [#43778](https://github.com/BerriAI/litellm/pull/43778) - refactor: clean up fresh tech debt from 2026-09-29 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43830](https://github.com/BerriAI/litellm/pull/43830) - test(router): settle the shared logging worker before recording shadow callbacks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43847](https://github.com/BerriAI/litellm/pull/43847) - chore(model_prices): add Gemini Veo, Mistral and Azure Claude 4.5 deprecation dates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43857](https://github.com/BerriAI/litellm/pull/43857) - fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43764](https://github.com/BerriAI/litellm/pull/43764) - refactor(rust): centralize Python bridge execution wrappers by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43871](https://github.com/BerriAI/litellm/pull/43871) - chore(cost-map): add openai gpt-image-2.5 batch prices from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43869](https://github.com/BerriAI/litellm/pull/43869) - chore(cost-map): add fireworks priority prices for ember-1, nemotron and glm 5.3 us rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43811](https://github.com/BerriAI/litellm/pull/43811) - fix(proxy): attribute completed batch cost rows to /batches in daily activity by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43870](https://github.com/BerriAI/litellm/pull/43870) - fix(router): strip encrypted reasoning the pinned deployment cannot decrypt by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43781](https://github.com/BerriAI/litellm/pull/43781) - fix(cost_calculator): stop copying optional_params into response hidden params by [@yucheng-berri](https://github.com/yucheng-berri) in [#43637](https://github.com/BerriAI/litellm/pull/43637) - feat(pricing): add vertex_ai gemini-3.8 flash tts rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43876](https://github.com/BerriAI/litellm/pull/43876) - fix(proxy): keep request-body credentials out of stored spend-log requests by [@yucheng-berri](https://github.com/yucheng-berri) in [#43635](https://github.com/BerriAI/litellm/pull/43635) - fix(ui): right-align money and count columns across tables by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#37889](https://github.com/BerriAI/litellm/pull/37889) - feat(agents): enforce authoritative agent permissions by [@joshua-berri](https://github.com/joshua-berri) in [#43721](https://github.com/BerriAI/litellm/pull/43721) - fix(proxy): strip caller credentials from websocket passthrough by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43855](https://github.com/BerriAI/litellm/pull/43855) - fix(ui): render access group MCP and agent selections as wrapping chips by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41228](https://github.com/BerriAI/litellm/pull/41228) - fix(responses): scan and mask top-level instructions with guardrails by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43629](https://github.com/BerriAI/litellm/pull/43629) - feat(traces): add Rust storage foundation by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43819](https://github.com/BerriAI/litellm/pull/43819) - feat(ui): agent traces tab on logs with timeline and otel setup guide by [@ishaan-berri](https://github.com/ishaan-berri) in [#43891](https://github.com/BerriAI/litellm/pull/43891) - feat(otel v2): excluded_services opt-out for datastore spans on tenant destinations by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43278](https://github.com/BerriAI/litellm/pull/43278) - fix(traces): correct ClickHouse rollup partitioning, dedupe keys, and retention changes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43901](https://github.com/BerriAI/litellm/pull/43901) - fix(bedrock): keep applicable beta headers by [@shrey-berri](https://github.com/shrey-berri) in [#43829](https://github.com/BerriAI/litellm/pull/43829) - feat(agents): authenticate Entra identities and delegated requests by [@joshua-berri](https://github.com/joshua-berri) in [#43722](https://github.com/BerriAI/litellm/pull/43722) - fix(hosted_vllm): keep reasoning_content on replayed assistant messages by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43599](https://github.com/BerriAI/litellm/pull/43599) - fix(proxy): delete large teams without per-member transaction fan-out by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42998](https://github.com/BerriAI/litellm/pull/42998) - feat(tracing): port OTLP ingestion to current trace foundation by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43915](https://github.com/BerriAI/litellm/pull/43915) - feat(proxy): add native ROI calculator for gateway spend vs merged PRs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43669](https://github.com/BerriAI/litellm/pull/43669) - test(e2e): repair completion, SAIL, and spend-log fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43902](https://github.com/BerriAI/litellm/pull/43902) - test(s3_v2): pin async 5xx retry through the production AsyncHTTPHandler by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43080](https://github.com/BerriAI/litellm/pull/43080) - feat(ui): adopt the new LiteLLM logo and monogram by [@yuneng-berri](https://github.com/yuneng-berri) in [#43913](https://github.com/BerriAI/litellm/pull/43913) - fix(proxy): relay Azure passthrough body model groups through the router by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43896](https://github.com/BerriAI/litellm/pull/43896) - fix(proxy): register a UI-configured arize callback next to otel under OTel v2 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43906](https://github.com/BerriAI/litellm/pull/43906) - fix(transcription): honor base_url alias for Groq Whisper and report it as the api base by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43917](https://github.com/BerriAI/litellm/pull/43917) - feat(tracing): store spend in ClickHouse automatically by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43928](https://github.com/BerriAI/litellm/pull/43928) - fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43903](https://github.com/BerriAI/litellm/pull/43903) - feat(agents): add identity registration and dashboard controls by [@joshua-berri](https://github.com/joshua-berri) in [#43723](https://github.com/BerriAI/litellm/pull/43723) - refactor(proxy): answer every team access check with TeamAccess.allows by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43364](https://github.com/BerriAI/litellm/pull/43364) - test(bedrock): restore the AWS env after a failed live call in the auth tests by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43921](https://github.com/BerriAI/litellm/pull/43921) - feat(lens): analyze agent activity with a separate worker by [@moe-berri](https://github.com/moe-berri) in [#43889](https://github.com/BerriAI/litellm/pull/43889) - fix(router): bill service tiers at catalog rates for custom-priced deployments by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43890](https://github.com/BerriAI/litellm/pull/43890) - test(ci): refresh qualified retired OpenAI fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43938](https://github.com/BerriAI/litellm/pull/43938) - fix(anthropic): forward the dangerous-tool-use beta to Azure AI Foundry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43934](https://github.com/BerriAI/litellm/pull/43934) - test(proxy): scope user_api_key_auth overrides in proxy_server tests by [@yuneng-berri](https://github.com/yuneng-berri) in [#43952](https://github.com/BerriAI/litellm/pull/43952) - fix(wandb): set supports_vision true on GLM-5.3-Flash by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43951](https://github.com/BerriAI/litellm/pull/43951) - chore(cost-map): sync openrouter prices from the models API by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43950](https://github.com/BerriAI/litellm/pull/43950) - fix(grayswan): send request conversation and tool calls to post-call monitor by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43770](https://github.com/BerriAI/litellm/pull/43770) - fix(azure_storage): name Data Lake objects without base64 padding or slashes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43914](https://github.com/BerriAI/litellm/pull/43914) - fix(guardrails): treat an unknown straiker api_version as unset instead of skipping the guardrail by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43956](https://github.com/BerriAI/litellm/pull/43956) - chore(deps): bump gitpython and tornado, extend diskcache osv ignore to Nov 1 by [@yuneng-berri](https://github.com/yuneng-berri) in [#43961](https://github.com/BerriAI/litellm/pull/43961) - fix(azure_storage): keep the DataLakeServiceClient alive until its TTL elapses by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43082](https://github.com/BerriAI/litellm/pull/43082) - feat(proxy): record in spend logs whether a request used a client-forwarded Anthropic OAuth token by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43063](https://github.com/BerriAI/litellm/pull/43063) - test(ci): repair stale tests and move retired OpenAI text-completion fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43958](https://github.com/BerriAI/litellm/pull/43958) - feat(e2e): record each e2e test's steps, starting with ProxyClient by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42393](https://github.com/BerriAI/litellm/pull/42393) - feat(providers): add Cortecs as an OpenAI-compatible provider by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43872](https://github.com/BerriAI/litellm/pull/43872) - feat(ui): filter tags by name and description on the Tag Management page by [@galovics](https://github.com/galovics) in [#42949](https://github.com/BerriAI/litellm/pull/42949) - fix(caching): write the response-cache SET to Redis at once instead of on the post-call batch by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43973](https://github.com/BerriAI/litellm/pull/43973) - test(e2e): typed per-test metadata for the e2e suite by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42044](https://github.com/BerriAI/litellm/pull/42044) - feat(guardrails): honor litellm_params.timeout in every HTTP guardrail by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43134](https://github.com/BerriAI/litellm/pull/43134) - chore(cost-map): add fireworks inkling priority prices from the prices api by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43949](https://github.com/BerriAI/litellm/pull/43949) - chore(cost-map): add deprecation date for anthropic claude-sonnet-4-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43898](https://github.com/BerriAI/litellm/pull/43898) - fix(cost-map): raise baseten DeepSeek-V4.1-Flash max output to 262144 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43916](https://github.com/BerriAI/litellm/pull/43916) - fix(proxy): restore pre-config-wins handling of pass-through endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#43962](https://github.com/BerriAI/litellm/pull/43962) - feat(lens): investigate sampled traces and retain batch results by [@moe-berri](https://github.com/moe-berri) in [#43942](https://github.com/BerriAI/litellm/pull/43942) - fix(guardrails): scan Responses API input in Azure Prompt Shield by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43786](https://github.com/BerriAI/litellm/pull/43786) - fix(guardrails): scan Responses API input in Azure Text Moderation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43965](https://github.com/BerriAI/litellm/pull/43965) - refactor: clean up fresh tech debt from 2026-09-30 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43993](https://github.com/BerriAI/litellm/pull/43993) - test: inject the H…

….105.0 ) (#3632) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.104.2` → `v1.105.0` | --- > ⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/3566) for more information. --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.105.0`](https://github.com/BerriAI/litellm/releases/tag/v1.105.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.104.3...v1.105.0) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.105.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.105.0/cosign.pub \ ghcr.io/berriai/litellm:v1.105.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - feat(mcp): share compatibility-aware result conversion across tool surfaces by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43089](https://github.com/BerriAI/litellm/pull/43089) - fix(caching): propagate auth cache invalidation over Redis Cluster via a node-level pub/sub client by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43110](https://github.com/BerriAI/litellm/pull/43110) - feat(model-catalog): add Rust registry validation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43136](https://github.com/BerriAI/litellm/pull/43136) - test(rust): reorganize core crate tests and split cache and OCR suites by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43177](https://github.com/BerriAI/litellm/pull/43177) - fix(proxy): give SpendLogToolIndex its own share of the cleanup budget and log a per-run summary by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41768](https://github.com/BerriAI/litellm/pull/41768) - feat(rust): hand upstream response headers to the native Messages stream by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43178](https://github.com/BerriAI/litellm/pull/43178) - feat(testkit): agent clients for Claude Code, Codex and opencode by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43181](https://github.com/BerriAI/litellm/pull/43181) - ci: run migrated unit selections on every event in legacy GHA shards by [@yuneng-berri](https://github.com/yuneng-berri) in [#43182](https://github.com/BerriAI/litellm/pull/43182) - test: move tests/test_litellm root and small trees into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43186](https://github.com/BerriAI/litellm/pull/43186) - fix(sso): gate /sso/debug routes behind ENABLE_SSO_DEBUG, off by default by [@ojensen-berri](https://github.com/ojensen-berri) in [#43150](https://github.com/BerriAI/litellm/pull/43150) - refactor(framer): replace Framer trait with tokio-util codecs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43193](https://github.com/BerriAI/litellm/pull/43193) - fix(anthropic): surface Responses bridge stream failures as Anthropic error events by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43126](https://github.com/BerriAI/litellm/pull/43126) - test: move tests/test_litellm/llms into tests/unit/llms by [@yuneng-berri](https://github.com/yuneng-berri) in [#43191](https://github.com/BerriAI/litellm/pull/43191) - test: move tests/test_litellm integrations and secret_managers into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43194](https://github.com/BerriAI/litellm/pull/43194) - feat(mcp): configure protocol versions and capability discovery by [@joshua-berri](https://github.com/joshua-berri) in [#43169](https://github.com/BerriAI/litellm/pull/43169) - fix(proxy): record streamed /v1/responses container ownership before the response.completed frame by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43140](https://github.com/BerriAI/litellm/pull/43140) - fix(tests): drop the repeated UNIT_FLAG key in test_unit_shard_missing_paths by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43212](https://github.com/BerriAI/litellm/pull/43212) - fix(router): count provider budget spend on every API surface by [@daqiangganjun](https://github.com/daqiangganjun) in [#38172](https://github.com/BerriAI/litellm/pull/38172) - fix(responses): fall back on pre-output stream drops, fail truncated streams, honor request_timeout by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43133](https://github.com/BerriAI/litellm/pull/43133) - fix(sentry): scrub PII and secrets inside object reprs and nested locals, add SENTRY_SEND_DEFAULT_PII opt-in by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43123](https://github.com/BerriAI/litellm/pull/43123) - fix(bedrock): surface a converse-stream 200 that decodes to no events as a 502 instead of an empty turn by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43213](https://github.com/BerriAI/litellm/pull/43213) - fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43206](https://github.com/BerriAI/litellm/pull/43206) - feat(integrations): add Databricks Zerobus trace logging callback by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42013](https://github.com/BerriAI/litellm/pull/42013) - fix(vertex_ai): keep legacy bucket_name in credential resolution and add GCS_BATCH_BUCKET_NAME env var by [@mubashir1osmani](https://github.com/mubashir1osmani) in [#42803](https://github.com/BerriAI/litellm/pull/42803) - feat(xai): add native xAI batches and files support by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42812](https://github.com/BerriAI/litellm/pull/42812) - fix(proxy): send a real error event when a /v1/messages stream fails by [@4refael](https://github.com/4refael) in [#41826](https://github.com/BerriAI/litellm/pull/41826) - fix(logging): redact raw_request when turn_off_message_logging is set in the proxy config by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43219](https://github.com/BerriAI/litellm/pull/43219) - fix(router): parse the classifier verdict out of surrounding prose instead of falling to the default tier by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43215](https://github.com/BerriAI/litellm/pull/43215) - test(zerobus): move tests into active CI selection by [@tin-berri](https://github.com/tin-berri) in [#43235](https://github.com/BerriAI/litellm/pull/43235) - fix(proxy): keep the submitted body out of 422 validation errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43231](https://github.com/BerriAI/litellm/pull/43231) - test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit by [@yuneng-berri](https://github.com/yuneng-berri) in [#43199](https://github.com/BerriAI/litellm/pull/43199) - chore(cost-map): add fireworks us-only deepseek v4.1 flash priority prices by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43247](https://github.com/BerriAI/litellm/pull/43247) - chore(cost-map): sync openrouter prices and add perceptron-mk1.5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43246](https://github.com/BerriAI/litellm/pull/43246) - fix(cost-map): correct fireworks_ai deepseek-v4p1-flash pricing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43253](https://github.com/BerriAI/litellm/pull/43253) - feat(openrouter): add typesafe/jev-router to the cost map by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43248](https://github.com/BerriAI/litellm/pull/43248) - chore(cost-map): add fireworks priority prices for muse glimmer 30b and deepseek v4 flash vision exp by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43252](https://github.com/BerriAI/litellm/pull/43252) - refactor(rust): move credential inheritance and SDK limits into a driver preflight by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43259](https://github.com/BerriAI/litellm/pull/43259) - fix(router): hold Responses lifecycle events until output so a pre-output fallback announces one response by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43238](https://github.com/BerriAI/litellm/pull/43238) - refactor(http): hand out an owned Client and route all providers through the pool by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43245](https://github.com/BerriAI/litellm/pull/43245) - test: fix stale and state-leaking tests red on scheduled CircleCI by [@yuneng-berri](https://github.com/yuneng-berri) in [#43266](https://github.com/BerriAI/litellm/pull/43266) - fix(cost-map): retirement dates, chatgpt reasoning flags, bing pricing, bedrock mantle and mythos, azure gpt-5.6 alias, anthropic batch rates, new nebius, openrouter and xai rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42951](https://github.com/BerriAI/litellm/pull/42951) - refactor(rust): promote anthropic messages out of experimental_pass_through by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43269](https://github.com/BerriAI/litellm/pull/43269) - test: stop CI tests from downloading tokenizer files and images by [@yuneng-berri](https://github.com/yuneng-berri) in [#43257](https://github.com/BerriAI/litellm/pull/43257) - fix(rust): preserve nested optional import failures by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43265](https://github.com/BerriAI/litellm/pull/43265) - ci: drop main and litellm\_\* branch filters from the CircleCI litellm-main workflows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43272](https://github.com/BerriAI/litellm/pull/43272) - feat(proxy): email alerts at configured percentages of a team member budget by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42665](https://github.com/BerriAI/litellm/pull/42665) - fix(callbacks-legacy-python): traverse and release the retained headers dict by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43274](https://github.com/BerriAI/litellm/pull/43274) - test(integration): port langfuse callbacks-in-db coverage to the local harness by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43282](https://github.com/BerriAI/litellm/pull/43282) - test(e2e): accept the otel cost write as a linked root trace by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42931](https://github.com/BerriAI/litellm/pull/42931) - test: finish the non-proxy half of tests/test_litellm by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43281](https://github.com/BerriAI/litellm/pull/43281) - refactor(rust): prepare inference and auth foundations for the gateway by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43287](https://github.com/BerriAI/litellm/pull/43287) - feat(rust): add config, router and gateway crates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43289](https://github.com/BerriAI/litellm/pull/43289) - fix(cost-map): remove duplicate openrouter/perceptron/perceptron-mk1.5 entry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43273](https://github.com/BerriAI/litellm/pull/43273) - fix(e2e): resolve the blank-S3 gateway repo root from the litellm package location by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42911](https://github.com/BerriAI/litellm/pull/42911) - test(integration): run the Langfuse DB-callback test on its own scratch database by [@yuneng-berri](https://github.com/yuneng-berri) in [#43288](https://github.com/BerriAI/litellm/pull/43288) - refactor(rust): expand logging and test coverage across gateway and Anthropic messages by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43295](https://github.com/BerriAI/litellm/pull/43295) - refactor: daily fresh tech debt cleanup, rolling PR (2026-09-25) by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43151](https://github.com/BerriAI/litellm/pull/43151) - test(proxy_behavior): scope the management proxy fixture to its package so its spend monitor cannot race the spend tests by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43302](https://github.com/BerriAI/litellm/pull/43302) - fix(cost-map): registry audit 2026-09-26, MAI-Image-2.5-Flash price, Databricks Claude Opus 5.5, Azure Foundry retirement dates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43254](https://github.com/BerriAI/litellm/pull/43254) - fix(cost-map): price fireworks deepseek v4.1 flash at the prices api value by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43311](https://github.com/BerriAI/litellm/pull/43311) - fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts by [@yuneng-berri](https://github.com/yuneng-berri) in [#43294](https://github.com/BerriAI/litellm/pull/43294) - fix(otel): detach post-response service spans by request phase, name redis spans by operation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43237](https://github.com/BerriAI/litellm/pull/43237) - test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test by [@yuneng-berri](https://github.com/yuneng-berri) in [#43321](https://github.com/BerriAI/litellm/pull/43321) - test(integration): pin the team-admin status-code matrix across every management route by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43249](https://github.com/BerriAI/litellm/pull/43249) - feat(proxy): add fail_closed_rate_limit_enforcement to reject requests with 503 while Redis rate limit counters are unreachable by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43251](https://github.com/BerriAI/litellm/pull/43251) - fix(responses): run stream failure and success hooks on the iterating loop instead of blocking it by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43270](https://github.com/BerriAI/litellm/pull/43270) - fix(guardrails): block private destinations in custom code http_request and bound guardrail execution time by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43280](https://github.com/BerriAI/litellm/pull/43280) - feat(sail): add Sail as a provider with service_tier mapped to its completion window by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42840](https://github.com/BerriAI/litellm/pull/42840) - test(mcp): pin server resolution and authorization behavior by [@joshua-berri](https://github.com/joshua-berri) in [#43261](https://github.com/BerriAI/litellm/pull/43261) - refactor(anthropic): rename experimental_pass_through to pass_through by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43329](https://github.com/BerriAI/litellm/pull/43329) - refactor(mcp): add shared server resolver without changing callers by [@joshua-berri](https://github.com/joshua-berri) in [#43262](https://github.com/BerriAI/litellm/pull/43262) - ci: fail on new unbounded SQL IN lists and add a Prisma chunking helper by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42629](https://github.com/BerriAI/litellm/pull/42629) - test(logging): drain the logging worker after each logging callback test so no later test inherits its events by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43344](https://github.com/BerriAI/litellm/pull/43344) - feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43271](https://github.com/BerriAI/litellm/pull/43271) - fix(mcp): adopt shared server resolution and caller authorization by [@joshua-berri](https://github.com/joshua-berri) in [#43263](https://github.com/BerriAI/litellm/pull/43263) - fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43022](https://github.com/BerriAI/litellm/pull/43022) - feat(proxy): add maximum_daily_tag_spend_retention_period cleanup setting by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#39221](https://github.com/BerriAI/litellm/pull/39221) - fix(cost-map): sync OpenRouter, Together, Cohere and Azure AI registry values with official sources by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43337](https://github.com/BerriAI/litellm/pull/43337) - docs(pr-template): add the backport-stable label only for a P0 regression by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43351](https://github.com/BerriAI/litellm/pull/43351) - fix(params): keep *litellm*\* kwargs out of provider request bodies by construction by [@shrey-berri](https://github.com/shrey-berri) in [#43221](https://github.com/BerriAI/litellm/pull/43221) - test: remove substring guard test_default_api_base by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43355](https://github.com/BerriAI/litellm/pull/43355) - ci: cut CircleCI wall time without loosening test isolation by [@yuneng-berri](https://github.com/yuneng-berri) in [#43347](https://github.com/BerriAI/litellm/pull/43347) - fix(cost-map): correct azure gpt-4o-mini tts, transcribe, alias and MAI-Image-2.5 prices by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43357](https://github.com/BerriAI/litellm/pull/43357) - fix(tests): stop VCR recording and replaying a test's own localhost upstream by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43346](https://github.com/BerriAI/litellm/pull/43346) - test(integration): group /v1/messages contracts under tests/integration/messages_endpoint by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43352](https://github.com/BerriAI/litellm/pull/43352) - fix(params): validate stream_chunk_size once, before any provider call by [@shrey-berri](https://github.com/shrey-berri) in [#43222](https://github.com/BerriAI/litellm/pull/43222) - fix(openai): exclude fine-tuned and custom gpt-5-chat aliases from gpt-5 reasoning path by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43185](https://github.com/BerriAI/litellm/pull/43185) - fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43322](https://github.com/BerriAI/litellm/pull/43322) - fix(caching): stand default cache points down when extra_body hides a direct client mark by [@yuneng-berri](https://github.com/yuneng-berri) in [#43341](https://github.com/BerriAI/litellm/pull/43341) - fix(mcp): report reachability without stored credentials by [@tin-berri](https://github.com/tin-berri) in [#43240](https://github.com/BerriAI/litellm/pull/43240) - fix(mcp): align hub publication status and controls by [@tin-berri](https://github.com/tin-berri) in [#43241](https://github.com/BerriAI/litellm/pull/43241) - chore(cost-map): sync openrouter prices for deepseek, minimax, qwen and glm rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43384](https://github.com/BerriAI/litellm/pull/43384) - chore(cost-map): drop stale cache hit field from openrouter deepseek-v4-pro-0813 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43389](https://github.com/BerriAI/litellm/pull/43389) - fix(responses): stream guardrail pre-call block as SSE with a typed output item by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42507](https://github.com/BerriAI/litellm/pull/42507) - feat(e2e): read management routes back from the control plane replicas by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43373](https://github.com/BerriAI/litellm/pull/43373) - feat(rust): add the openai_like chat config foundation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43379](https://github.com/BerriAI/litellm/pull/43379) - feat(router): opt in to prompt-cache cost routing by [@tin-berri](https://github.com/tin-berri) in [#43232](https://github.com/BerriAI/litellm/pull/43232) - feat(otel): add SigNoz preset for OpenTelemetry v2 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43296](https://github.com/BerriAI/litellm/pull/43296) - feat(cli): reuse saved agent setup and add reconfigure by [@tin-berri](https://github.com/tin-berri) in [#43392](https://github.com/BerriAI/litellm/pull/43392) - fix(streaming): keep litellm Usage on text-completion usage chunks by [@yuneng-berri](https://github.com/yuneng-berri) in [#43047](https://github.com/BerriAI/litellm/pull/43047) - fix(vertex_ai): consider tools when validating context caching min tokens by [@agustin18](https://github.com/agustin18) in [#43319](https://github.com/BerriAI/litellm/pull/43319) - fix(anthropic): drop thinking blocks with empty thinking text, not just missing signature by [@shoemoney](https://github.com/shoemoney) in [#38049](https://github.com/BerriAI/litellm/pull/38049) - test(e2e): report batch cleanup leftovers as a plain UserWarning by [@yuneng-berri](https://github.com/yuneng-berri) in [#43405](https://github.com/BerriAI/litellm/pull/43405) - fix(tools): salvage concatenated JSON tool call arguments by [@kumarpriyanshu09](https://github.com/kumarpriyanshu09) in [#43260](https://github.com/BerriAI/litellm/pull/43260) - fix(gemini): forward seed to the Gemini API instead of rejecting it by [@YaseenBashaT](https://github.com/YaseenBashaT) in [#43197](https://github.com/BerriAI/litellm/pull/43197) - fix(vertex_ai): make Gemma fake streams work with traced Responses by [@stewartpark](https://github.com/stewartpark) in [#43147](https://github.com/BerriAI/litellm/pull/43147) - fix(responses): emit the reasoning item on streaming /v1/responses for signature-only thinking by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43414](https://github.com/BerriAI/litellm/pull/43414) - fix(anthropic): forward the per-turn-control beta to Azure AI Foundry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43415](https://github.com/BerriAI/litellm/pull/43415) - test(vertex_ai): assert the usage the Gemma responses stream actually reports by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43422](https://github.com/BerriAI/litellm/pull/43422) - fix(bedrock): keep the provider status code on unprocessable image errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43416](https://github.com/BerriAI/litellm/pull/43416) - fix(vertex_ai): stop importing the vertexai SDK in partner-model completion by [@anmolg1997](https://github.com/anmolg1997) in [#42274](https://github.com/BerriAI/litellm/pull/42274) - fix(token_counter): count Gemini function_declarations tools by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43417](https://github.com/BerriAI/litellm/pull/43417) - refactor(types): replace Any with proven types in 5 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43304](https://github.com/BerriAI/litellm/pull/43304) - chore(cost-map): update azure_ai/grok-4.6 input price from Azure pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43440](https://github.com/BerriAI/litellm/pull/43440) - chore(cost-map): add azure_ai/MAI-Cyber-1-Flash by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43446](https://github.com/BerriAI/litellm/pull/43446) - refactor(rust): share anthropic types, request helpers, and streaming contracts across crates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43426](https://github.com/BerriAI/litellm/pull/43426) - feat(rust): expand gateway configuration parsing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43460](https://github.com/BerriAI/litellm/pull/43460) - refactor(rust): share call lifecycle across route-owned inference by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43461](https://github.com/BerriAI/litellm/pull/43461) - feat(rust): add the HTTP host driver by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43462](https://github.com/BerriAI/litellm/pull/43462) - build(rust): package the gateway container by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43471](https://github.com/BerriAI/litellm/pull/43471) - refactor(rust): use shared execution in gateway inference by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43463](https://github.com/BerriAI/litellm/pull/43463) - fix(cost-map): add deprecation_date to together_ai Salesforce/Llama-Rank-V1 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43507](https://github.com/BerriAI/litellm/pull/43507) - feat(rust): support the HTTP Responses API by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43464](https://github.com/BerriAI/litellm/pull/43464) - fix(cost-map): sync openrouter prices from the models API by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43506](https://github.com/BerriAI/litellm/pull/43506) - fix(cost-map): set together_ai gpt-oss-20b and gemma-4-31B-it deprecation_date to 2026-09-15 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43509](https://github.com/BerriAI/litellm/pull/43509) - feat(rust): add litellm-db and litellm-db-testing workspace scaffolding by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43504](https://github.com/BerriAI/litellm/pull/43504) - feat(rust): connect Python inference bindings to shared routes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43465](https://github.com/BerriAI/litellm/pull/43465) - feat(rust): add structured route lifecycle tracing by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43466](https://github.com/BerriAI/litellm/pull/43466) - feat(rust): separate gateway authentication and authorization by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43467](https://github.com/BerriAI/litellm/pull/43467) - feat(rust): add virtual key storage contracts by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43468](https://github.com/BerriAI/litellm/pull/43468) - feat(rust): add gateway UI login and sessions by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43469](https://github.com/BerriAI/litellm/pull/43469) - feat(rust): add the MCP gateway by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43470](https://github.com/BerriAI/litellm/pull/43470) - refactor(rust): remove delivery routing abstraction by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43514](https://github.com/BerriAI/litellm/pull/43514) - feat(azure): add mistral ocr pricing and azure max output limits by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43530](https://github.com/BerriAI/litellm/pull/43530) - refactor: clean up fresh tech debt from 2026-09-27 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43538](https://github.com/BerriAI/litellm/pull/43538) - refactor(types): replace Any with proven types in 8 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43551](https://github.com/BerriAI/litellm/pull/43551) - test(rust): enforce shared upstream error contract in wheel checks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43520](https://github.com/BerriAI/litellm/pull/43520) - fix(proxy): log key owner identity on expired key auth failures by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43105](https://github.com/BerriAI/litellm/pull/43105) - fix(cost-map): registry audit 2026-09-28, openai deep-research shutdown dates, azure deepseek v4.1 flash direct price, vertex gemini 3.8 live avatar price, drop azure_ai/muse-spark-1.3 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43566](https://github.com/BerriAI/litellm/pull/43566) - security(proxy): keep team callback credentials out of the stored request body by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43217](https://github.com/BerriAI/litellm/pull/43217) - feat(model_prices): add claude-sonnet-5-5 model pricing by [@krrish-berri-2](https://github.com/krrish-berri-2) in [#43586](https://github.com/BerriAI/litellm/pull/43586) - test(integration): pin org-admin status codes in the team-admin matrix by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43592](https://github.com/BerriAI/litellm/pull/43592) - refactor(guardrails): fix agent 365 to the production endpoint and log the opt-in fail_open at error level by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43189](https://github.com/BerriAI/litellm/pull/43189) - fix(model_prices): correct Claude Sonnet 5.5 capabilities and provider keys by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43587](https://github.com/BerriAI/litellm/pull/43587) - test(unit): stop test modules from putting their own directory on sys.path by [@yuneng-berri](https://github.com/yuneng-berri) in [#43421](https://github.com/BerriAI/litellm/pull/43421) - refactor(rust): centralize host execution and compose callbacks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43515](https://github.com/BerriAI/litellm/pull/43515) - fix(proxy): unregister logging callbacks removed from the stored config by [@yuneng-berri](https://github.com/yuneng-berri) in [#43428](https://github.com/BerriAI/litellm/pull/43428) - chore(cost-map): take azure limits for deepseek-v4-flash-0731 and v3.2-speciale by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43597](https://github.com/BerriAI/litellm/pull/43597) - revert: "feat(proxy): server-side Team Usage export beyond the top-N key cap ([#42996](https://github.com/BerriAI/litellm/issues/42996))" by [@yassin-berriai](https://github.com/yassin-berriai) in [#43376](https://github.com/BerriAI/litellm/pull/43376) - fix(cost-map): add web search flag and model page source to anthropic claude-sonnet-5-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43584](https://github.com/BerriAI/litellm/pull/43584) - refactor(mcp): consolidate hub publication predicate by [@tin-berri](https://github.com/tin-berri) in [#43394](https://github.com/BerriAI/litellm/pull/43394) - fix(provider): accept 2xx status codes in unified_access_group create by [@Louis-Vauterin](https://github.com/Louis-Vauterin) in [#42461](https://github.com/BerriAI/litellm/pull/42461) - fix(panw_prisma_airs): honor experimental_use_latest_role_message_only on every request shape by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42447](https://github.com/BerriAI/litellm/pull/42447) - fix(cost-map): add Vertex batch cache prices to vertex_ai/claude-sonnet-5-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43602](https://github.com/BerriAI/litellm/pull/43602) - revert: "feat(usage): search team keys beyond the top-N in the Team usage view ([#42857](https://github.com/BerriAI/litellm/issues/42857))" by [@yassin-berriai](https://github.com/yassin-berriai) in [#43377](https://github.com/BerriAI/litellm/pull/43377) - feat(providers): add Prism provider (internal copy of [#40914](https://github.com/BerriAI/litellm/issues/40914)) by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41961](https://github.com/BerriAI/litellm/pull/41961) - test(integration): credential canary suite harness by [@yucheng-berri](https://github.com/yucheng-berri) in [#43300](https://github.com/BerriAI/litellm/pull/43300) - feat(bedrock): add xai grok-4.7 pricing and sync llama, mistral large 2407 and minimax m2.5 prices by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43623](https://github.com/BerriAI/litellm/pull/43623) - ci: sync the weekly release cycle with Linear releases by [@yuneng-berri](https://github.com/yuneng-berri) in [#43636](https://github.com/BerriAI/litellm/pull/43636) - feat(cache): select Rust caching through explicit cache objects by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43601](https://github.com/BerriAI/litellm/pull/43601) - test(integration): credential canary slots for MCP and pass-through credentials by [@yucheng-berri](https://github.com/yucheng-berri) in [#43308](https://github.com/BerriAI/litellm/pull/43308) - test(integration): stored-config credential canary slots by [@yucheng-berri](https://github.com/yucheng-berri) in [#43309](https://github.com/BerriAI/litellm/pull/43309) - fix(guardrails): preserve Presidio output selection and restoration by [@joshua-berri](https://github.com/joshua-berri) in [#43401](https://github.com/BerriAI/litellm/pull/43401) - fix(ui): rename All Models tab to Deployed Models and model filters to All Proxy Models by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43638](https://github.com/BerriAI/litellm/pull/43638) - feat(mcp): scan and pin upstream tool descriptions by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43283](https://github.com/BerriAI/litellm/pull/43283) - feat(cost-map): add bedrock_mantle rows for claude opus 5.5 and sonnet 5.5 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43647](https://github.com/BerriAI/litellm/pull/43647) - chore(codeowners): drop UI, migration, and CODEOWNERS self owners by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43653](https://github.com/BerriAI/litellm/pull/43653) - feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43641](https://github.com/BerriAI/litellm/pull/43641) - feat: add model leaderboard page by [@ishaan-berri](https://github.com/ishaan-berri) in [#43649](https://github.com/BerriAI/litellm/pull/43649) - fix(google_genai): preserve proxy_server_request in completion adapter by [@hsm207](https://github.com/hsm207) in [#43536](https://github.com/BerriAI/litellm/pull/43536) - fix(otel): send cache and reasoning tokens in langfuse usage_details by [@ankit373](https://github.com/ankit373) in [#43553](https://github.com/BerriAI/litellm/pull/43553) - fix(vertex_ai): forward the per-turn-control beta for per-message output_config by [@DeviaVir](https://github.com/DeviaVir) in [#43558](https://github.com/BerriAI/litellm/pull/43558) - fix(cost-map): add tool calling and reasoning flags, correct max output for nebius DeepSeek-V4.1-Flash by [@Flexomatic81](https://github.com/Flexomatic81) in [#43588](https://github.com/BerriAI/litellm/pull/43588) - fix(proxy): resolve model_group_alias in the zero-cost budget predicate by [@fedaeho](https://github.com/fedaeho) in [#43512](https://github.com/BerriAI/litellm/pull/43512) - refactor: clean up fresh tech debt from 2026-09-28 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43674](https://github.com/BerriAI/litellm/pull/43674) - refactor(types): replace Any with proven types in 7 files by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43704](https://github.com/BerriAI/litellm/pull/43704) - docs(security): point readers to the security announcements mailing list signup by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43713](https://github.com/BerriAI/litellm/pull/43713) - refactor(rust): add shared llms wire type derives by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43730](https://github.com/BerriAI/litellm/pull/43730) - fix(router): stream /v1/messages lifecycle frames live when no fallback can take over by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43600](https://github.com/BerriAI/litellm/pull/43600) - feat(cost-map): add baseten DeepSeek-V4.1-Flash-Fast by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43735](https://github.com/BerriAI/litellm/pull/43735) - feat(guardrails): send a configured gateway_name from noma_v2 to Noma by [@itaimodi](https://github.com/itaimodi) in [#43678](https://github.com/BerriAI/litellm/pull/43678) - chore(cost-map): add openai gpt-6.1-sol from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43738](https://github.com/BerriAI/litellm/pull/43738) - test(integration): callback credential canary slots C1-C3 and D5 by [@yucheng-berri](https://github.com/yucheng-berri) in [#43630](https://github.com/BerriAI/litellm/pull/43630) - test(integration): request-path credential canary slots D1-D4 by [@yucheng-berri](https://github.com/yucheng-berri) in [#43307](https://github.com/BerriAI/litellm/pull/43307) - test(integration): sweep proxy logs, metrics, a Datadog intake and the Logs drawer for credential canaries by [@yucheng-berri](https://github.com/yucheng-berri) in [#43306](https://github.com/BerriAI/litellm/pull/43306) - chore(cost-map): add azure and openrouter gpt-6.1-sol rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43744](https://github.com/BerriAI/litellm/pull/43744) - test(proxy): classify every credential-bearing param for the canary suite by [@yucheng-berri](https://github.com/yucheng-berri) in [#43298](https://github.com/BerriAI/litellm/pull/43298) - fix(cost-map): lower fireworks up-to-4b size tier to the pricing page price by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43740](https://github.com/BerriAI/litellm/pull/43740) - fix(cost_calculator): bill chat per-second pricing once with a new cost_per_second field by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43614](https://github.com/BerriAI/litellm/pull/43614) - chore(cost-map): add openai gpt-6-astra ultrafast tier prices from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43745](https://github.com/BerriAI/litellm/pull/43745) - fix(autorouter): compare historical and new savings consistently by [@tin-berri](https://github.com/tin-berri) in [#43348](https://github.com/BerriAI/litellm/pull/43348) - fix(model-prices): align Azure, Bedrock, Copilot, Gemini, Groq, OpenAI and OpenRouter entries with official docs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43598](https://github.com/BerriAI/litellm/pull/43598) - fix(guardrails): enable explicit PANW MCP output scanning by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43109](https://github.com/BerriAI/litellm/pull/43109) - refactor(rust): orchestrate Messages route execution by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43719](https://github.com/BerriAI/litellm/pull/43719) - fix(streaming): keep the served service_tier on streamed chunks and spend rows by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42870](https://github.com/BerriAI/litellm/pull/42870) - feat(bedrock): add openai gpt-6.1-sol global and base rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43758](https://github.com/BerriAI/litellm/pull/43758) - perf(router): fetch cooldown state and usage counters in one Redis round trip by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43320](https://github.com/BerriAI/litellm/pull/43320) - perf(proxy): hold one spend counter batch across admission and across post-call accounting by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43369](https://github.com/BerriAI/litellm/pull/43369) - perf(responses): run aresponses through the async wrapper so the cache is read once by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43769](https://github.com/BerriAI/litellm/pull/43769) - fix(router): bind Claude Code background sessions to their auto-router by [@tin-berri](https://github.com/tin-berri) in [#43767](https://github.com/BerriAI/litellm/pull/43767) - feat(bedrock): add openai.gpt-6.1-sol us geo cris and Mantle rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43763](https://github.com/BerriAI/litellm/pull/43763) - fix(proxy): recover session key owners from daily spend for usage attribution by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43642](https://github.com/BerriAI/litellm/pull/43642) - chore(cost-map): take azure context limits from models-sold-directly by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43759](https://github.com/BerriAI/litellm/pull/43759) - perf(proxy): one request-scoped Redis pipeline for auth, spend, rate-limit and routing reads by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43407](https://github.com/BerriAI/litellm/pull/43407) - fix(mcp): scope OpenAPI listings to the exact server prefix and drop upstream OAuth metadata when a server is saved by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43608](https://github.com/BerriAI/litellm/pull/43608) - perf(proxy): one post-call Redis pipeline per backend for spend, rate-limit, routing and response-cache writes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43779](https://github.com/BerriAI/litellm/pull/43779) - fix(bedrock): set gpt-6.1-sol max output tokens to 131072 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43782](https://github.com/BerriAI/litellm/pull/43782) - perf(proxy): refresh auth management objects through the request Redis pipeline by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43776](https://github.com/BerriAI/litellm/pull/43776) - feat(agents): add identity storage and validation contracts by [@joshua-berri](https://github.com/joshua-berri) in [#43720](https://github.com/BerriAI/litellm/pull/43720) - chore: bump litellm-enterprise 0.1.71 -> 0.1.72, litellm-proxy-extras 0.4.102 -> 0.4.103, litellm 1.104.0 -> 1.105.0 by [@yuneng-berri](https://github.com/yuneng-berri) in [#43789](https://github.com/BerriAI/litellm/pull/43789) - fix(auth): give UI/CLI session tokens their own AES-GCM context and header-safe shape by [@yuneng-berri](https://github.com/yuneng-berri) in [#43790](https://github.com/BerriAI/litellm/pull/43790) - chore(deps): bump pyjwt, moment and brace-expansion to clear osv-scan by [@yuneng-berri](https://github.com/yuneng-berri) in [#43792](https://github.com/BerriAI/litellm/pull/43792) - fix(ui): surface x-litellm-call-id in Logs search, table and drawer by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42436](https://github.com/BerriAI/litellm/pull/42436) - fix(proxy): look up hashed key names with two spend log rows per key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43656](https://github.com/BerriAI/litellm/pull/43656) - fix(cost-map): add deprecation_date to two together_ai nvidia rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43809](https://github.com/BerriAI/litellm/pull/43809) - test(ci): refresh retired OpenAI tool-call models by [@yuneng-berri](https://github.com/yuneng-berri) in [#43676](https://github.com/BerriAI/litellm/pull/43676) - fix(ui): keep MCP permissions visible after key, team and MCP server saves by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43810](https://github.com/BerriAI/litellm/pull/43810) - test(ci): repair MCP Responses and budget fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43788](https://github.com/BerriAI/litellm/pull/43788) - test(bedrock): accept regional aliases that inherit Converse routing by [@yuneng-berri](https://github.com/yuneng-berri) in [#43785](https://github.com/BerriAI/litellm/pull/43785) - fix(params): categorize internal param appropriately to prevent leaking into request by [@shrey-berri](https://github.com/shrey-berri) in [#43783](https://github.com/BerriAI/litellm/pull/43783) - perf(router): honour the cooldown read interval in the routing prefetch by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43815](https://github.com/BerriAI/litellm/pull/43815) - fix(router): carry per-request routing reads on context variables instead of public method kwargs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43814](https://github.com/BerriAI/litellm/pull/43814) - fix(bedrock): add beta header for output config in message by [@shrey-berri](https://github.com/shrey-berri) in [#43778](https://github.com/BerriAI/litellm/pull/43778) - refactor: clean up fresh tech debt from 2026-09-29 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43830](https://github.com/BerriAI/litellm/pull/43830) - test(router): settle the shared logging worker before recording shadow callbacks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43847](https://github.com/BerriAI/litellm/pull/43847) - chore(model_prices): add Gemini Veo, Mistral and Azure Claude 4.5 deprecation dates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43857](https://github.com/BerriAI/litellm/pull/43857) - fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43764](https://github.com/BerriAI/litellm/pull/43764) - refactor(rust): centralize Python bridge execution wrappers by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43871](https://github.com/BerriAI/litellm/pull/43871) - chore(cost-map): add openai gpt-image-2.5 batch prices from the pricing page by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43869](https://github.com/BerriAI/litellm/pull/43869) - chore(cost-map): add fireworks priority prices for ember-1, nemotron and glm 5.3 us rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43811](https://github.com/BerriAI/litellm/pull/43811) - fix(proxy): attribute completed batch cost rows to /batches in daily activity by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43870](https://github.com/BerriAI/litellm/pull/43870) - fix(router): strip encrypted reasoning the pinned deployment cannot decrypt by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43781](https://github.com/BerriAI/litellm/pull/43781) - fix(cost_calculator): stop copying optional_params into response hidden params by [@yucheng-berri](https://github.com/yucheng-berri) in [#43637](https://github.com/BerriAI/litellm/pull/43637) - feat(pricing): add vertex_ai gemini-3.8 flash tts rows by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43876](https://github.com/BerriAI/litellm/pull/43876) - fix(proxy): keep request-body credentials out of stored spend-log requests by [@yucheng-berri](https://github.com/yucheng-berri) in [#43635](https://github.com/BerriAI/litellm/pull/43635) - fix(ui): right-align money and count columns across tables by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#37889](https://github.com/BerriAI/litellm/pull/37889) - feat(agents): enforce authoritative agent permissions by [@joshua-berri](https://github.com/joshua-berri) in [#43721](https://github.com/BerriAI/litellm/pull/43721) - fix(proxy): strip caller credentials from websocket passthrough by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43855](https://github.com/BerriAI/litellm/pull/43855) - fix(ui): render access group MCP and agent selections as wrapping chips by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#41228](https://github.com/BerriAI/litellm/pull/41228) - fix(responses): scan and mask top-level instructions with guardrails by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43629](https://github.com/BerriAI/litellm/pull/43629) - feat(traces): add Rust storage foundation by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43819](https://github.com/BerriAI/litellm/pull/43819) - feat(ui): agent traces tab on logs with timeline and otel setup guide by [@ishaan-berri](https://github.com/ishaan-berri) in [#43891](https://github.com/BerriAI/litellm/pull/43891) - feat(otel v2): excluded_services opt-out for datastore spans on tenant destinations by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43278](https://github.com/BerriAI/litellm/pull/43278) - fix(traces): correct ClickHouse rollup partitioning, dedupe keys, and retention changes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43901](https://github.com/BerriAI/litellm/pull/43901) - fix(bedrock): keep applicable beta headers by [@shrey-berri](https://github.com/shrey-berri) in [#43829](https://github.com/BerriAI/litellm/pull/43829) - feat(agents): authenticate Entra identities and delegated requests by [@joshua-berri](https://github.com/joshua-berri) in [#43722](https://github.com/BerriAI/litellm/pull/43722) - fix(hosted_vllm): keep reasoning_content on replayed assistant messages by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43599](https://github.com/BerriAI/litellm/pull/43599) - fix(proxy): delete large teams without per-member transaction fan-out by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42998](https://github.com/BerriAI/litellm/pull/42998) - feat(tracing): port OTLP ingestion to current trace foundation by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43915](https://github.com/BerriAI/litellm/pull/43915) - feat(proxy): add native ROI calculator for gateway spend vs merged PRs by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43669](https://github.com/BerriAI/litellm/pull/43669) - test(e2e): repair completion, SAIL, and spend-log fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43902](https://github.com/BerriAI/litellm/pull/43902) - test(s3_v2): pin async 5xx retry through the production AsyncHTTPHandler by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43080](https://github.com/BerriAI/litellm/pull/43080) - feat(ui): adopt the new LiteLLM logo and monogram by [@yuneng-berri](https://github.com/yuneng-berri) in [#43913](https://github.com/BerriAI/litellm/pull/43913) - fix(proxy): relay Azure passthrough body model groups through the router by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43896](https://github.com/BerriAI/litellm/pull/43896) - fix(proxy): register a UI-configured arize callback next to otel under OTel v2 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43906](https://github.com/BerriAI/litellm/pull/43906) - fix(transcription): honor base_url alias for Groq Whisper and report it as the api base by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43917](https://github.com/BerriAI/litellm/pull/43917) - feat(tracing): store spend in ClickHouse automatically by [@yujonglee-berri](https://github.com/yujonglee-berri) in [#43928](https://github.com/BerriAI/litellm/pull/43928) - fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43903](https://github.com/BerriAI/litellm/pull/43903) - feat(agents): add identity registration and dashboard controls by [@joshua-berri](https://github.com/joshua-berri) in [#43723](https://github.com/BerriAI/litellm/pull/43723) - refactor(proxy): answer every team access check with TeamAccess.allows by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#43364](https://github.com/BerriAI/litellm/pull/43364) - test(bedrock): restore the AWS env after a failed live call in the auth tests by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43921](https://github.com/BerriAI/litellm/pull/43921) - feat(lens): analyze agent activity with a separate worker by [@moe-berri](https://github.com/moe-berri) in [#43889](https://github.com/BerriAI/litellm/pull/43889) - fix(router): bill service tiers at catalog rates for custom-priced deployments by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43890](https://github.com/BerriAI/litellm/pull/43890) - test(ci): refresh qualified retired OpenAI fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43938](https://github.com/BerriAI/litellm/pull/43938) - fix(anthropic): forward the dangerous-tool-use beta to Azure AI Foundry by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43934](https://github.com/BerriAI/litellm/pull/43934) - test(proxy): scope user_api_key_auth overrides in proxy_server tests by [@yuneng-berri](https://github.com/yuneng-berri) in [#43952](https://github.com/BerriAI/litellm/pull/43952) - fix(wandb): set supports_vision true on GLM-5.3-Flash by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43951](https://github.com/BerriAI/litellm/pull/43951) - chore(cost-map): sync openrouter prices from the models API by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43950](https://github.com/BerriAI/litellm/pull/43950) - fix(grayswan): send request conversation and tool calls to post-call monitor by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43770](https://github.com/BerriAI/litellm/pull/43770) - fix(azure_storage): name Data Lake objects without base64 padding or slashes by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43914](https://github.com/BerriAI/litellm/pull/43914) - fix(guardrails): treat an unknown straiker api_version as unset instead of skipping the guardrail by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43956](https://github.com/BerriAI/litellm/pull/43956) - chore(deps): bump gitpython and tornado, extend diskcache osv ignore to Nov 1 by [@yuneng-berri](https://github.com/yuneng-berri) in [#43961](https://github.com/BerriAI/litellm/pull/43961) - fix(azure_storage): keep the DataLakeServiceClient alive until its TTL elapses by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43082](https://github.com/BerriAI/litellm/pull/43082) - feat(proxy): record in spend logs whether a request used a client-forwarded Anthropic OAuth token by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43063](https://github.com/BerriAI/litellm/pull/43063) - test(ci): repair stale tests and move retired OpenAI text-completion fixtures by [@yuneng-berri](https://github.com/yuneng-berri) in [#43958](https://github.com/BerriAI/litellm/pull/43958) - feat(e2e): record each e2e test's steps, starting with ProxyClient by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42393](https://github.com/BerriAI/litellm/pull/42393) - feat(providers): add Cortecs as an OpenAI-compatible provider by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43872](https://github.com/BerriAI/litellm/pull/43872) - feat(ui): filter tags by name and description on the Tag Management page by [@galovics](https://github.com/galovics) in [#42949](https://github.com/BerriAI/litellm/pull/42949) - fix(caching): write the response-cache SET to Redis at once instead of on the post-call batch by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43973](https://github.com/BerriAI/litellm/pull/43973) - test(e2e): typed per-test metadata for the e2e suite by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#42044](https://github.com/BerriAI/litellm/pull/42044) - feat(guardrails): honor litellm_params.timeout in every HTTP guardrail by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43134](https://github.com/BerriAI/litellm/pull/43134) - chore(cost-map): add fireworks inkling priority prices from the prices api by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43949](https://github.com/BerriAI/litellm/pull/43949) - chore(cost-map): add deprecation date for anthropic claude-sonnet-4-5 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43898](https://github.com/BerriAI/litellm/pull/43898) - fix(cost-map): raise baseten DeepSeek-V4.1-Flash max output to 262144 by [@berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#43916](https://github.com/BerriAI/litellm/pull/43916) - fix(proxy): restore pre-config-wins handling of pass-through endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#43962](https://github.com/BerriAI/litellm/pull/43962) - feat(lens): investigate sampled traces and retain batch results by [@moe-berri](https://github.com/moe-berri) in [#43942](https://github.com/BerriAI/litellm/pull/43942) - fix(guardrails): scan Responses API input in Azure Prompt Shield by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43786](https://github.com/BerriAI/litellm/pull/43786) - fix(guardrails): scan Responses API input in Azure Text Moderation by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43965](https://github.com/BerriAI/litellm/pull/43965) - refactor: clean up fresh tech debt from 2026-09-30 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#43993](https://github.com/BerriAI/litellm/pull/43993) - test: inject the HIBP client and the MCP loop clock so two backend tests stop flaking by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#&#…

devin-ai-integration Bot requested a review from a team September 26, 2026 02:07

devin-ai-integration Bot assigned mateo-berri Sep 26, 2026

mateo-berri added the run-ci label Sep 26, 2026

greptile-apps Bot reviewed Sep 26, 2026

View reviewed changes

Comment thread litellm/responses/streaming_iterator.py

mateo-berri added run-ci and removed run-ci labels Sep 26, 2026

greptile-apps Bot reviewed Sep 26, 2026

View reviewed changes

github-advanced-security Bot found potential problems Sep 26, 2026

View reviewed changes

Comment thread litellm/responses/streaming_iterator.py Fixed

fix(responses): submit the sync failure handler only after the async …

0be4a64

…one finishes

mateo-berri added run-ci and removed run-ci labels Sep 26, 2026

cursor Bot reviewed Sep 26, 2026

View reviewed changes

mateo-berri approved these changes Sep 26, 2026

View reviewed changes

mateo-berri merged commit 5ac640e into main Sep 26, 2026
148 of 158 checks passed

mateo-berri deleted the litellm_responses_stream_failure_handlers_on_loop branch September 26, 2026 19:22

mateo-berri mentioned this pull request Oct 14, 2026

fix(responses): retain strong ref to streaming success-logging tasks (GC race) #31074

Open

Conversation

devin-ai-integration Bot commented Sep 26, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Before (4adbc13)

Mid-stream provider drop on /v1/responses

Drop while six other requests' log flushes hold the logger

Healthy stream completes

Mid-stream drop on /v1/chat/completions through the Responses bridge

Mid-stream drop on /v1/messages through the Responses bridge

Drop while one earlier request's log flush holds the logger (the repro's frozen-pod sequence, three attempts)

After (0be4a64)

Mid-stream provider drop on /v1/responses

Drop while six other requests' log flushes hold the logger

Healthy stream completes

Mid-stream drop on /v1/chat/completions through the Responses bridge

Mid-stream drop on /v1/messages through the Responses bridge

Drop while one earlier request's log flush holds the logger (the repro's frozen-pod sequence, three attempts)

Type

Decisions

Caveats (if any)

Low

Live A/B risk check

Final Attestation

Uh oh!

devin-ai-integration Bot commented Sep 26, 2026 • edited by mateo-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

Uh oh!

codecov Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

mateo-berri commented Sep 26, 2026

Uh oh!

mateo-berri commented Sep 26, 2026

Uh oh!

Uh oh!

greptile-apps Bot Sep 26, 2026

Choose a reason for hiding this comment

Uh oh!

mateo-berri Sep 26, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mateo-berri commented Sep 26, 2026

Uh oh!

mateo-berri commented Sep 26, 2026

Uh oh!

mateo-berri commented Sep 26, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

devin-ai-integration Bot commented Sep 26, 2026 •

edited by cursor Bot

Loading

Before (`4adbc13`)

After (`0be4a64`)

devin-ai-integration Bot commented Sep 26, 2026 •

edited by mateo-berri

Loading

greptile-apps Bot commented Sep 26, 2026 •

edited

Loading

codecov Bot commented Sep 26, 2026 •

edited

Loading

codspeed Bot commented Sep 26, 2026 •

edited

Loading